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INTRODUCi iON 

Modern spacecraft represent a significant investment. The 
return on this investment can only be realized if these craft 
adequately perform the assigned missions in spite of the strin- 
gent environment of outer space and unexpected circumstance. 

The latest advances in engineering technology are brought to 
bear to overcome the challenge posed by the environment, and 
yet the designer recognizes his inability to foresee all 
possible circumstance. Therefore, to compensate for the 
unexpected, the control of modern spacecraft remains largely in 
human hands. 

Attempts have been made to automate those lower level 
routine functions that must be performed within the system. The 
human operator is only called upon to manage these by exception... 
to assume control whenever there is an indication of inadequate 
performance in some particular regard. He can compensate for 
failures by reconfiguring the mechanism, or if necessary, 
altering the mission. The higher levels of decision making 
remain solely the prerogative of the human operator. The need 
for close control of the developing situation dramatizes the 
potential value of having man aboard the spacecraft. 

Yet two important missions stand in need of taking new 
steps toward autonomous spacecraft. The first of these concerns 
exploration of deep space. Here, it is clearly unsuitable to 
include the human operator aboard the vehicle. Further, remote 
control becomes inadequate in view of the significant communica- 
tion delays. Such spacecraft must be intelligently capable of 
evaluating a variety of opportunities and coping with unexpected 
threats, for only then can the mission be completed with the 
greatest degree of success. 

Sc long as the intent of the exploration is defined a 
strict sense of what is sought, the findings a *e unduly r •- 
strained. The value of having a man onboard l.ir(,;ly rest- ..pon 
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his ability to understand the mission in some broader sense so 
that he can take advantage of "targets of opportunity" and modify 
the mission to explore previously unforeseen avenues. By the 
same token, he would be in a position to more properly evaluate 
unanticipated dangers and, recognizing these, alter the course of 
the mission or take other appropriate actions. In essence, the 
challenge is to devise logical processes that can perform this 
sophisticated function. We must incorporate a decision-making 
mechanism that simulates some of the essential features of human 
intelligence, at least to the extent of referencing the broad 
scope of the mission intent and optimal selection of response 
behavior in the light of that purpose and the developing situation. 

The second mission concerns the retrieval of spacecraft 
that require refurbishment or may have failed in orbit. The 
space shuttle is equipped to accomplish this for low altitude 
target objects. It seems reasonable to expect similar retrieval 
devices for synchronous orbit target objects in the near future. 

Control of the retrieval mechanism requires prediction of 
the relative orientation and motion of the target object, thus 
making it possible to safely approach, contact, and bring that 
object aboard the shuttle. But, the target object may behave 
in an erratic manner. Contact with it may cause a change of 
its internal state and perhaps activation of its propulsion 
system. Further, the retrieval mechanism may behave in a complex 
manner when coupled with such a target object. Even if the 
target is passive, perfect reliability of the retrieval mechanism 
is never assured. It is therefore essential to design for 
"graceful failure" in that inappropriate retrieval might prove 
disasterous for the shuttle. 

Both of these missions encwmpa:.s the more general problem 
wherein it's desirable to approach and investigate or evadp ^onip 
particular object in space. Some target objects of intcre..!, 
might be less well-known than our own failed satellites. The 
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very presence of the "shuttle" may affect the behavior cf such an 
object . . . causing the object to adopt a collision course 
(through a gravitational attraction or as a result of the pro- 
grammed propulsion of a space mine). Alternatively, the object 
might take an evasive course (in the case of a foreign satellite 
programmed to avoid being captured). 

The situation becomes even more complex if the target 
object can operate at some higher level of intelligence. For 
example, it may be purposive within some context that includes 
the friendly spacecraft. Note that if there is adequate remote 
monitoring and close control capability, this intelligence need 
not be onboard the target object. In the case of space explora- 
tion, a number of friendly spacecraft may be assigned to 
cooperate by performing complementary functions. Here their 
interactive behavior re quires gaming in an effective manner so 
that they benefit one another and operate collectively to best 
support the accomplishment of the mission. Under certain circum- 
stances a single spacecraft of the consort might be assigned a 
terminal mission, this in order to gain and transfer the 
knowledge required to increase the likelihood of success for 
those that remain. 

The purpose of this investigation has been to explore some 
ways in which autonomous behavior can be extended to treat situ- 
ations wherein close control by the human operator may not be 
appropriate or even possible. 



DISCUSSION 


Intellicient behavior begins with holding a concise under- 
standing of what is to be accomplished. Ordinarily, purpose is 
depicted only in terms of the most desirable outcome. . . with 
some consideration being given to those alternative futures that 
are considered most undesirable. In point of fact, purpose 
becomes well defined if, and only if, it consists of a statement 
of the relative worth of each of the significantly different 
futures . . . this being expressed in the form of a hierarchic 
valuated state space and appropriate normalizing function. 

Intelligent behavior also requires an adequate sensing 
system. There is some chance that the environment will be as 
desired, but it is more likely to experience the contrary. 
Opportunities may be some distance off-course, and there may be 
stumbling blocks or greater dangers directly in the way. The 
sensor system must allow observation in such a way as to enable 
pattern recognition. . . re-cognition, knowing again what has 
been known before. Simply stated, pattern recognition consists 
of comparing the observed environment to similar templates 
referenced from memory. A decision is made as to which of these 
templates is most like the present observation. Note that this 
process yields only a limited understanding ... one restricted 
to the vocabulary of templates; most endeavors to improve pattern 
recognition are concerned with selecting a most suitable nearness 
metric and finding convenient means for computing the error. 

More sophisticated purposive behavior involves classifica- 
tion . . . the discovery of useful templates. Here the intent 
is to characterize the observed environment in terms of the 
existing regulations. Mathematical techniques in clustering, 
factor analysis, and discriminant function analysis provide 
meaningful ways to group data poi .ts within a predefined sto^e 
space. Here again, the task is ' define that spare in tern's ct 
axes and a distance metric. 
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Decisions are made in sequence. It is therefore essential 
to find temporal regularities and extend these for the sake of 
predicting the environment. Purposive behavior hinges on an 
ability to predict the environment in order to anticipate oppor- 
tunities and avoid threats. Lacking an ability to predict one's 
environment generally precludes intelligent behavior. 

AUhough the process of prediction can be identified with 
the scientific method, it is more convenient to briefly state 
that forecasting requires definition of prediction span (the time 
of future concern), an ability to retrieve the sensed data from 
memory, and a criterion that specifically indicates the relative 
worth of each of the possible correct and incorrect predictions. 
This allows developing a model of those regularities that can then 
be extended to yield a most . appropri ate forecast on the basis of 
what is known and the criterion of predictive performance. Note 
that this process requires pattern recognition in the sense that 
the recorded data base must be referenced in terms of the given 
criterion (payoff mat^-ix, error cost function, predictive goal). 
It also requires classification in the sense that the regular- 
ities already experienced must be identified before these can be 
considered in consort and collectively extended to yield a fore- 
cast. Indeed, the process of prediction is nect'’sarily inductive 
and therefore cannot be performed with perfect certainty. 
Efficient structuring of a useful model is the very essence of 
creativi ty . 

Although the literature is replete with numerous methods for 
prediction, most of these treat the process of forecasting only 
with respect to the least mean squared error criterion. This 
tradition has grown in view of analytic procedures which are made 
far more tractable with this criterion. For example. Regression 
Analysis and Fourier Analysis provide methods for forecasting 
time series on the basis of the least mean squared error 
criterion. But in the real world, equally c:rrect i nternref r o- 
are not of equal worth, and the various ori'os of forcca T. i 
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usually are attributed widely different costs. Appendix A 
indicates a specific method for extracting cyclic components from 
an arbitrary environment with respect to the usual criterion and 
with respect to an arbitrary criterion. Such a method is 
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essential for treating environments wherein the best prediction 
is not simply the most likely future, but rather that future 
which reflects the underlying purpose of the prediction. 


The credibility of any predictive model can only be 
determined by examining the validity of its forecasts over time. 

In general , the same model is used again and again so long as its 
predictions are of sufficient worth. If, on the other hand, the 
model proves untrue (that is, sufficiently costly), its credi- 
bility is degraded. It then becomes worthwhile to introduce 
other uncorrelated data for the sake of generating a new model 
worthy of testing. In point of fact, models (theories, conjec- 
tures, hypotheses, rules, laws, and so forth) must include infor- 
mation beyond that contained within the data base. Ordinarily, 
such additional information comes from prior experience using 
different types of models in similar problem domains. Without 
such learning, the creative process is reduced to a selection 
upon randomness. 

The credibility of each predictive model can be estimated 
by comparing the forecast of that model to the most recently 
experienced data. It is tempting, but invalid, to extend the 
mathematical model into the recent past to yield postdictions 
to be compared point by point with respect to the error criterion, 
for here the same data base is being used twice. The proper pro- 
cedure is to truncate the data at, say, a point in recent time 
comparable in span to the time interest of the forecast, then 
generate a new model based on the truncated data, then evaluate 
its nredictions against the recent past. Presumably, the same 
kind • model would have a similar credibility. 

Prediction is a basis for rontrol. Fach precltcu 

based on a model that represcr»-s the underlying logic c* he 
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environment. If the environment is responsive to snini. ion, 
then a prediction of its response in the light of a recent 
sequence of stimuli is based on a model of the transduction. 
Control theory was developed with the intent of causing a linear 
plant to behave in a desired manner. Some treatment was then 
given to certain particular nonlinearities in that all real world 
transduction is nonlinear. The problem of identification arises 
when it becomes necessary to characterize an unknown plant. More 
carefully stated, the task is to select from the available 
resources that stimulus which is most likely to cause the unknown 
environment to yield the desired response. And this is a sim- 
plistic view, for it is important to understand when the specific 
desire cann ot be realized and, if it cannot be, then what stim- 
ulus is most desirable in the sense of yielding a worthwhile, if 
not most desirable, response. Modern control theory does not 
treat this problem in complete generality but rather offers par- 
ticular approximations on the basis of limiting assumptions. In 
general, there should be no such presumption. The unknown plant 
may be linear or nonlinear, passive or active, possibly even 
intelligently cooperative, ambivalent or competitive. 

Evolutionary programming provides a general approach for 
prediction and control in this regard. Preliminary findings on 
such programming have been reported in the 1 iterature. However, 
a new program was written wherein finite state machines are 
scored in their ability to predict each data point in the most 
recent portion of the experienced data stream, this with respect 
to an arbitrary criterion. An original machine is chosen at 
random or on the basis of assumptions concerning the underlying 
regularities within the environment. This parent machine is then 
mutated in a random manner to yield an offspring which is then 
similarly scored in terms of its ability to forecast each next 
point in the recent past. If the score for this offspring is 
less than that of the parent, that offspring is discorded, and 
a new offspring is generated. If, howevf’r . th- rfsprinq is 
superior -.c the parent, this o^*:. pring bei_.M..' s a now pat 1 . 1. 
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Such nonregressi ve evolution proceeds In fast time until a 
higher level criterion is reached. For example, the predictive 
file may reach a sufficient level, or the computational time or 
space may run out. The resulting machine is then exercised to 
yield the required forecast. 

This prediction is then compared with the actual next state 
of the environment, and the question is raised as to the next 
symbol. Here the machine used for the last prediction becomes 
the progenitor of the next evolutionary exploration, for surely 
some useful regularity must have been found, even if the current 
prediction may be in error. 

It is convenient to include a cost for complexity in the 
structure indicating the worth of each evaluated machine. In 
essence, this embodies the Maxim of Parsimony. If this factor is 
small, the evolving machines grow in complexity to express each 
regular aspect of the environment in the light of the criterion. 
If this factor is large, the machines are reduced to an over- 
simplified view of these regularities. Note that a periodic 
environment of arbitrary cycle can be perfectly represented by a 
single state machine. The program written for such evolutionary 
prediction and modeling permits a variable alphabet size, arbi- 
trary predictive criteria and includes an inner loop scored in 
terms of its ability to forecast eaclt next point in the recent 
past. If the score for this offspring is less than that of the 
parent, that offspring is discarded, and a new offspring is 
generated. If, however, the offspring is superior to the parent, 
this offspring becomes a new parent. Such nonregressi ve evolu- 
tion proceeds in fast time until a higher level criterion is 
reached. For example, the predictive file may reach a sufficient 
level, or the computational time or space may run out. The 
resulting machine is then exercised to yield the required 
forecast . 

This prediction is then compared with the actual next state 
of the environment, and the question is raised as to the next 

Cr.iC: ' . C! '-r. 

OF POOR QUALITY 


8 



J 


OF POC^x 

symbol. Here the machine used for the last predictioi bec^niis 
the progenitor of the next evolutionary exploration, for sur? ly 
some useful regularity must hav^ been found, even that allows the 
nature of the mutation noise to be a function of the prior suc- 
cess of that kind of noise in the evolutionary process. This 
method for prediction provides a significant advantage with 
respect to modeling and closing the control loop for an arbitrary 
envi ronment . 

Each prediction is compared with the next actual output to 
yield a measure of the credibility of the identification process. 
When sufficient credibility has been reached, the model of the 
plant can be used as a basis for closing the loop in an appro- 
priate manner. In essence, successful prediction confirms the 
model as a replica of the plant. 


face ]S 
quautv 



Here is a critical aspect of control loop design and yet a 
straightforward logic permits determination of each next optimal 
resource assignment. The logic references the control goal 
(a valuated state space that portrays each of the significantly 
different futures and their relative worth), the allocable 
resources at that moment in time, and the finite-state machine 
feedback from the predictor, reference Figure 1. 


Examine only the present state of that machine. Each of the 
transitions from that state is examined in an exhaustive manner 
to determine if any transition indicates the more desired output 
from the plan. If so, the related input is noted, and there is 
a test of the inventory of resources to determine if such a 
stimulus can be invoked to yield the desired response. If either 
the output symbol or the required resource is not found, refer- 
ence is made to the next most desirable state in terms of the 
corresponding output symbol and required resource assignment. 

The process continues until a commitment is mede or tnere is a 
determination that no meaningful options are ; pen . For ov^rmle, 
if the model is the machine shown in Figure 2 , then ref.-enc? to 
the present state, K, indicates that an output of four c'nnot be 
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obtained, regardless of the input. The desire is to have the 
largest output response, then obviously this is nine, provided 
five is an allocable resource. Note that, in general, the plant 
may be controllable only in certain states and even controllable 
to a different degree as a function of the state. Here is the 
essence of the control theory without the usual restricting 
assumpti ons . 

The problem of control occurs in many different regimes. 

The task of retrieving objects from space by means of the shuttle 
is difficult for several reasons. First, the retrieval arm is 
not a rigid body. The equations of motion for such a flexible 
structure are quite complex. Nevertheless, modern structural 
analysis programs on large digital computers provide a means for 
performing structural analysis by finite element methods, taking 
only the first few bending modes into account. 

Second, the shuttle does not provide a stable base in 
inertial space. Therefore, the geometric problems associated 
with the ret'ieval of an object from space are an order of 
magnitude more complicated than those faced by designers of ter- 
restrial robots, where it is reasonable to expect an inertially 
fixed base for the robot. 

Third, the exact size and mass of the object to be retrieved 
(the spacecraft) may not be known; and therefore, a prediction of 
the motion once the object to be retrieved has been "grabbed" by 
the retrieving system may not be possible with the desired degree 
of reliability for a successful retrieval operation. 

Fourth, the initial motion, and therefore the initial move- 
ment and moment of momentum of the spacecraft, may be unknown or 
only approximately known. It is also not known if the spacecraft 
is spinning, whether it may be retrieved while in that. sL.tL<- '.r 
first the motion must be reduce ' some degree, o * ’t ay ' *'• 
be stopped completely (at some additional ro«>t in si/e ai- i , 
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Fifth, the forces and moven-ints acting on t * spacecraft 
not be known. Frequently, of Cvjrse, t spaceoift is a pas- 
sive body, exerting no forces or moveirients by itself. On the 
other hand, it may be a satellite with an attitude control system 
v.hich may or may not be functioning properly. One could even con- 
sider the case of a satellite whose attitude control system may 
have been struck, so that it might seem that the satellite is 
passive, but the shock of the capture might make it active once 
again. 

These five areas of uncertainties {and there are more, such 
as temperature effects and others) should indicate that an off- 
line study and simulation of a particular retrieval task will 
generally provide only a baseline model from which first-order 
approximations to guidance laws may be derived and around which 
certain sensitivity studies may be performed. If the total 
amount of the uncertainties is relatively small, and if no 
unforeseen effects take place during the retrieval process, such 
an off-line model and simulation may be adequate. If, however, 
many uncertainties exist, a real-time systems' identification 
and an adaptive control system may be required to prevent a 
catastrophic failure. 

There exists many ways to address the problem of adaptive 
control and on-line system identification. In some cases, one 
assumes a certain structure or topology of the system (for 
example, a linear, second or higher order system with constant 
coefficients), and the on-line identification process then merely 
consists of estimating these coefficients such that a certain 
error (usually in some least mean square sense) between 
observed and modeled behavior of the system is minimized. In the 
case at hand, such a parameter might be the mass, or the moment 
of inertia, or the rotational frequency of the spacecraft. 

Taking a radically different approach, tie spacecraft is 
not moceled as a linear system, nor as a system (i«>sf,r i he<) a 
set of linear or nonlinear diffeiential equations, bet ; . iu r , 
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in terms of finite state machines. These machines may be used 
to perform any one of the following three functions: prediction 
of each next input symbol from a sequence of observed past 
symbols; transformation of a sequence of input symbols into a 
sequence of output symbols; or classification of a g'ven 
sequence of symbols. All three of these functions may be used 
in the control of the retrieval operation. The first property 
may be used to predict the motion (position, attitude, and their 
derivatives) of the spacecraft and of the end effector. The 
transformation property can be used to effect the control of the 
actuators of the retriever. Finally, the classification 
prop rty may be helpful to identify certain classes of spacecraft. 

Various approaches may be taken to create finite state 
machines capable of performing these three tasks, the most 
general one is the method of evolutionary programming. That it 
is, indeed, feasible to use finite state machines for tasks of 
prediction and identification in control systems has previously 
been shown by key personnel of Decision Science, Inc.* 

Using finite-state machines as a controller is, of course, 
not a panacea that resolves all the space shuttle's control 
problems. One of the difficulties when using finite-state 
machines lies in the alphabet size, which is a reflection of the 
required resolution. In the past, evolutionary programmers 
have created finite-state machines within an alphabet of up to 
64 symbols. This corresponds with a resolution of 6 bits. For 
identification and control tests, machines with a resolution of 
8 to 9 bits appear to be desirable, thus, the alphabet size 
would be 256 or 512. 

■’’his gives an angular resolution of about one degree. The 
problem with such a large alphabet size is the long past history 


" F i n i te- State Machines as Clements in Control Systems" 
by G. Burqin and M. Walsh, 1971, IEEE Systems, Man and 
Cybernetics Group, Convention Record, IEEE 1971, pp, 241-246. 


'.2 


0^.*-= ' ' . *vv 

.. - V ; ». • ^ *' 

f . • ' 

required to evolve finite-state machines. This means that con- 
siderable resources are required in terms of memory and CPU 
capability. In the past, such large alphabet sizes were unecon- 
omical; but with the present availability of 16-bit micropro- 
cessors. and the recent introduction of INTELS iAPX432 chips, 
which provide, with only four chips, a 32-bit processor compar- 
able to a medium sized IBM 370 model. CPU capability should be 
no limitation for an on-line identification and adaptive control 
using evolutionary programming. There remains the question of 
sufficient memory. Here again. 64k bit RAMs are now available, 
and magnetic bubble memories allow even greater densities. 

About a half-dozen manufacturers will have bubble memories 
commercially available by the end of this year. 

Advantage should be taken of this ; lenomenal progress in 
hardware, both CPU and memory, together with a drastic reduction 
in size and weight, to solve the space shuttle's retrieval 
problems. Evolutionary programming is a powerful method but was 
previously limited in its applicability to real-life problems 
because of hardware limitations. Now, it seems, the time has 
come to demonstrate the usefulness of this method. 

Automatic control of robotic devices requires the solution 
of two problems. first, the goal of the robotic device must be 
defined, and that requires a precise formulation of what the 
robot is supposed to do. This is a problem pertaining to the 
field of artificial intelligence. If the robotic device is 
simply a mechanical manipulator, the question reduces to: "To 

which point in space and along which path is the robofs end 
effector supposed to move, and what should its ..Icitude be at 

vhe terminal point?" 

The second problem attempts to find forces and moments 
acting on the manipulator such that it will perform the motion 
specified above. In other words, the question is: "How do we 

force the robot to do what it is supposed to do?" This is a 
problem of applied automatic ccutrol. 
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Appendix C of this report addresses this problem. First, 
the different control schemes, which have been proposed in the 
recent open literature, are reviewed. Then, a novel scheme 
of controlling robots of unknown physical properties is pro- 
posed wherein the control system's gains are adjusted by means 
of finite-state machines. To develop such a control system, 
a computer simulation of a manipulator is required. Appendix C 
offers a detailed account of such a simulation for a specific, 
commercially available manipulator, the PUMA 250. A control 
system capable of controlling this manipulator's upper and 
lower arm from any arbitrary point in space to any other, 
physically reachable point, is designed. The response of this 
control system is quite satisfactory for payloads ranging from 
zero to five times nominal payload. 

Briefly stated, prediction is the basis for control. The 
predictive model is a first cut representation of the logic 
underlying the environment of interest. Predicting the envir- 
onmental response to the sequence of stimuli allows the predic- 
tion process to forecast each next response, but more impor- 
tantly, this yields an up-to-date representa t i on of the trans- 
duction. In other words, prediction of each response based on 
prior stimulus/response pairs can be used to resolve the identi- 
fication problem. Once this is accomplished, the remaining 
task is to determine how to close the loop. 

The literature is replete with techniques for the control of 
linear and certain non-linear systems with respect to relativelv 
simple control goals (the criterion that specifies the worth 
of each correct control response and the cost of the alternative 
errors). The problem remains unresolved for the general situa- 
tion of a non-linear, potentially active, and even intelligent 
plant to be controlled with respect to an arbitrary control goal. 
The present task is to explore a generalized technique in this 
regard . 
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Having a capability to model the environment sets the stage 
for self-modeling. Here the same process may be applicable. But, 
there must be recognition of fundamental limitations concerning 
sel f- referential systems. Sel f-di agnosti c routines already exist. 

In a sense, these are sel f- referenti al but limited in scope to the 
designer's prior knowledge of the alternatives that might arise. 
More sophisticated "self-awareness" requires continual modeling of 
the self as exhibited under the developing situation together with 
an ability to reference such models for the sake of improving the 
likelihood of correct response in the goal seeking interaction 
with the environment. To be meaningful, such "consciousness" must 
be sufficiently precise (that is of adequate specificity), suffi- 
ciently accurate (demonstrate a continuing correspondence with 
reality), adaptable (capable of updating as new aspects of the 
self come into view), and readily referencable (suitable for 
immediate data retrievable and updating in view of the required 
response time). Once such a capability has been realized, it is 
of interest to enquire as to the possibility of still higher level 
modeling of the self. Here the process is analogous to knowing 
that you exist and knowing that you know that you exist. With 
such an onboard capability, spacecraft may become more adaptive, 
taking into account not only the changing environment but their 
own remaining capabilities. Such artificial consciousness sets 
the stage for intelligent interaction among spacecraft and other 
autonomous inanimate entities. 

The purpose of another entity can only be inferred from its 
behavior and the presumption of similarity between the "organisms". 
Such higher level modeling sets the stage for interacting with 
other intelligent creatures in a meaningful manner. Note that 
the game can be played at different levels at the same time. 

"What resource am I willing to expend to learn more about the 
other player's goal so that I might better direct my efforts in 
future moves?" A more detailed understanding of this process 


should provide new Insight into the nature of coalition building 
and generalized gaming. 

Lastly, the essential concern for the very mechanism of 
purpose cannot be avoided. How does purpose arise? Is there a 
minimum complexity of logical structure required for the gene»'a- 
tion of the survival "instinct" . . . the paramount purpose of 
all living creatures? The approach to this problem requires a 
careful definition of the nature of purpose and reference to 
logical means for generating such representations within 
mechanisms. It seems reasonable to suspect that inanimate 
machines can be designed having sufficient complexity to generate, 
then seek their own purposes. The "human animal" is a demonstra- 
tion of one such mechanism. The task remaining is to understand 
the logic that allows such mechanisms to operate. An 
understanding in this regard might open the door to the design of 
highly sophisticated spacecraft that construct and deploy 
"offspring". Monitoring their activity should prove to be 
instructive . 
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CONCLUSION 

It is easy to envision the requirement for retrieval, 
repair, and replacement of satellites. Here prediction and 
control processes are required. These can be approached by 
classical means, but it is time to examine an alternative 
approach. . . one that can treat an arbitrary data stream, 
predicting each next data point with respect to an arbitrary 
payoff matrix. The criterion for linking spacecraft is more 
complex than least mean squared error. Final closure is at 
great cost if, say, the angular difference is excessive. 

Success is assured only if the angular difference is within the 
acceptable bounds. The task is to predict closure windows with 
just such a criterion in mind. 

The problem of control can also be addressed by classical 
means; however, it might be more suitable to represent the 
alternative states of the closing vehicles and determine each 
next move by reference to the prospective transition among such 
states. We learn to ride a bicycle not by solving the equations 
of motion but by remembering how best to actuate the controls 
from the given situation to reach a more desirable state. 
Computation for control of the shuttle might be benefited in 
this same regard. 

Finite-state machines provide a natural means for repre- 
senting the logic which may underlie a sequence of data derived 
from a sensed environment or for depicting the transduction 
between stimulus and response of such an environment. Such 
representation permits expansion of the logic in terms of 
arbitrary input and output languages so long as these are 
expressed within finite alphabets. Further, the machines may be 
of arbitrary specificity so long as they have only a finite 
number of states. Thus, no unnatural constraint is imposed, as 
is so often the case when a sequence of data is expressed in 
terms of a linear difference or differential equation. 
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Exploratory spacecraft require onboard computation that 
predicts dangers and opportunities, then responds to these 
through appropriate control actions. Clearly, it is desirable 
to avoid colliding with meteors and other space debris. It is 
essential to avoid space mines that may seek to collide on the 
basis of IR, radar, or other sensed information. It is desir- 
able to come sufficiently close and interact with various 
interesting objects or regions so that these can be suitably 
investigated. Such interaction may involve the cooperative 
construction of space based platforms or other facilities. It 
may involve close control onboard the robots with strategic 
control reserved to the human operator, or in the more distant 
future, autonomous spacecraft interacting with each other on 
thei r own behalf. 

Decision Science, Inc. 

San Diego, California, June 1982 
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APPENDIX A 

DETERMINING THE PERIODS OF NOISY SEQUENTIAL DATA 

I INTRODUCTION 

Many phenomena give rise to periodic sequences of data 
values. Economic indicators, celestial events, human 
processes, electrical circuits, tumbling spacecraft, --the 
list is endless. Given an arbitrary sequences of data 
values, the problem is to determine the smallest period. If 
the data is "noisy," that is, if the sequence is only 
approximately periodic, then the task becomes more 
interesting and the result more practical. The purpose here 
is to explicate in detail a general procedure for 
determining those periodic sequences which best fit an 
arbitrary sequence of data values. Statistical measures are 
developed by which to define this best periodic sequence and 
for determining confidence limits on the error when 
subsequent values in the given data sequence are estimated 
by the corresponding values in the periodic sequence. 

II THE SMALLEST PERIOD OF A PERIODIC SEQUENCE 
Suppose that X(l), X(2), ..., X(n) is a finite 

sequence of n real numbers. By definition, the sequence 
(X) is periodic with period p . if and only if for all 
positive integers k, X(k + - X(k). Equi volent 1 /. Tv., 

all non-negative integers i and all positive mteg?’ ^ ><' 

between 1 and p inclusive. 


X(k + pi) 


X(k). 
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In other words, if (1 £ k p) then 

X(k) = X(k + p) = X(k + 2p) = ... 

If both p and q are periods of the sequence (X) 
then it follows that (p,q), the greatest common positive 
divisor of the integers p and q is also a period of (X). 
To show this recall that if d is the greatest common 
divisor of p and q, then there exist two integers, a 
and b, such that 

d = pa + qb. 

Therefore, for all positive integers k, 

X(k + d) = X(k + pa + qb) . 

Since d is positive, a and b cannot both be negative 
and so either (k + pa) or (k + qb) is positive. If 
k + pa, say, is positive, then 

X(k + pa + qb) = X(k pa) 

because (X) has period q. But since p is also a 
period, 

X(k + pa) = X(k) 

Therefore for all positive integers k, 

X(k + d) * X(k) 

A-2 
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That Is, (X) has period d. 

This implies that there is a smallest period for a 
given sequence (X) and that this smallest period is a 
divisor of all other periods for (X). That is, all other 
periods are multiples of the smallest period. It suffices, 
therefore, to determine the smallest period of a sequence, 
all other periods being multiples. 

Ill THE METHOD 

If (X) is exactly periodic with smallest period p, 
then there are just p data values to the sequence, 
subsequent values being repetitions. These values are X(l), 
X(2), ... , X(p) and they constitute one cycle of the 
periodic sequence. If on the other hand (X) is merely 
approximately periodic, then, for example, the data values 
X(l), X(1 + p), X(1 + 2p), ... vill only be approximately 
equal. Similarly for (1 < k < p), the values X(k), 

X(k + p), X(k + 2p), ... will only be approximately equal. 
X(l), X(I + p), X(1 + 2p), ... are all the first data 
values of all the cycles; similarly X(k), X(k + p), 

X(k + 2p), ... are all the kth data values of the cycles 
of the sequence. 

The method used here to determine a periodic sequence 
(Y) that best fits the given approximately periodic 
sequence (X) is to determine, for each possible period u. 
the average (arithmetic mean), Y(k), nf all the kth data 
values of the cycles of the sequence (X). That is. 


OR!G‘r: : r ^ J 
, . OF POO.* Qw U'TY 

Y(k) = — i-T S X(k + pi) 

T + 1 i = 0 

where T is the number of cycles. Here, T can be 
expressed in closed form as 

T = Int((n - k)/p), 

where Int(j) is the greatest integer less than or equal to j 

j. This formula for T takes into consideration the 
likelihood that the finite data sequence (X) may end in a 
partial cycle. 

In order to measure the degree of fit of the periodic 
sequence (Y) to the sequence (X), the sample variance, 

V(k), for each of the p values of the cycle 
(k = 1, 2, p) is determined by 

T 

V(k) = (1/T) D [x(k + pi) - Y(k)]^ 

Assuming that T is at least 30 or that deviations of the 
data values X(k + pi), (i = 0, 1, 2, ..., T are normally 

% 

distributed, it is then appropriate to calculate 95X 
confidence error bounds for Y(k) when Y(k) is used as an 
estimate of any individual value X(k + pi), i = 1, 2, ..., 

T , ... 

Depending upon the particular application, there are 
various criteria for determinmu the bes> i‘i- pe. ioJi.. 
sequence Y(k). For many pu*"p-^'''S the st^nd*rr 
squares criterion is appropriate and so this cr to-ion t s 
A-4 
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developed first. But it is easy to imagine situations 
where, for example, the cost of error is asymmetric, thus 
making it optimal to fit the sequence (X) with a periodic 
sequence (Y) which is not optimal in the least squares 
sense. This more general case is considered later. 

The most straightforward criterion for determining the 
best fit periodic sequence (Y) is the average 95 % 
confidence error tolerance, E, for the p values Y(k), 
k = 1, 2, ..., p. E is given by 

P 

E = (1/p) L E(k) , 

k = 1 

where, by well-known statistical methods, 

E(k) = .. .WMk) . 

>/(T + 1) 

t being the appropriate t- distribution value for T 
degrees of freedom. 

Each possible period p generates a periodic sequence 
(Y) and an average 95% confidence error bound E for (Y) 
That sequence (Y) whose average error bound E is 
smallest is deemed the best least squares fit periodic 
sequence approximating the given sequence (X). 

IV COMPUTER PROGAM 

The procedure oitlir.ed ' ne oreviou*; woi>- 

hardly be practical without tie aid of a higi spec* 


[ 
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computer. (Indeed many procedures once thought impractical 
can now be resurrected and applied in placo of more analytic 
techniques. ) 

A computer program has been written and demonstrated 
that takes an arbitrary sequence of real njmbers (X) and 
determines the ten best periods and corresponding periodic 
sequences (Y). Ninety-five percent confidence error bounds 
are given for each data value Y(k), k * 1, 2, .... p. As 
indicated previously, the best periodic sequence (Y) is 
the one whose average error bound over the p values of any 
one cycle is minimal. 

The program is interactive and allows convenient input, 
storage, recall, and display of all relevant parameters. 

(See pages A - 12 - 19) . 

V SAMPLE RUN OF THE COMPUTER PROGRAM 

When the program is executed, the user is asked an 
initial question. By entering a single letter the user may 
1) input a new sequence of real numbers, or 2) recall from 
storage a previously input data sequence, or 3) list the 
data sequence, or 4) correct individual members of the data 
sequence, or 5) analyze the data sequence for 
periodicities, or 6) display this period analysis, or 7) 
store the data sequence for future retrieval, or 8) end the 
sess i on . 

If the user wishes to input a data sequence nntc-c 

the appropriate letter, then valuer a<*e accepted in 
A-6 
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of 20 or less. After data input is completed, the program 
returns to the initial question. 

At this time individual data values may be listed and 
corrected by entering the appropriate letter and then 
entering the particular index and corresponding sequence 
value. Multiple corrections can be made without return to 
the initial question. 

Return to the initial question allows the user to 
analyze the data sequence for periodicities. When the 
analysis is completed, the total number of data v •. lues is 
given together with the period which best fits the data, the 
number of complete cycles and the predicted next value in 
the data sequence. The user is now asked whether he wishes 
to analyze a particular potential period, or rank the 
periods according to goodness of fit or end the display and 
return to the initial question. If the user wishes to 
analyze some particular period p, he enters that period and 
quickly sees a table containing an in-depth analysis 
Including error estimates and statistical measures. The 
user may then immediately analyze any other period. 

By entering a single letter the user may also display a 
table ranking all possible periods from 1 to 10 according to 
their average 95X confidence error. This is tie average 
error when sequence values are estimated by the associated 
periodic sequence corstructed by avenging t'le vartoii'; 
representat i ves comirg froi:i Jifferv.nt _y,ics of th.. ,it:. 
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After analysis and display is completed, the user may 
store the data sequence for future retrieval. 

Three different data sequences were generated to test 
the program. (See the end of this appendix.) The program 
determined the best fit periodic sequences and successfully 
carried through the other steps explicated in the previous 
paragraphs of this section. 


VI OTHER CRITERIA FOR OPTIMALITY 

One of the advantages of the method outlined above is 
that the criterion for optimality can be easily generalized 
without extensive alteration of the procedure. 

Suppose that instead of the least squares criterion, 
there is defined a cost function, C, that assigns to each 
ordered pair (x.y) the worth of estimating y when in 
fact the actual value is x. Clearly, C will have its 
smallest entries on the diagonal, where the estimate is 
exactly correct. Off-diagonal entries may be arbitrarily 
assigned depending upon the context. C may be quite non- 
symmetr i c . 

In the least squares case, the average 

T 

Y(k) = ■ - ■ ■ L X(k + ip) 

T + 1 i s 0 


can be easily shown to be the best estimate of the data 
values X(k + -p), i « 0 1 T. 9ut for an arbitrary 

cost function C, the best ett’mat'' ♦o'- e ^ 
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depends upon C. Therefore it is necessary first to 
determine Y(k) for each k = 1, 2, .... p by finding that 
value Y(k) for which the average cost 

T 

F(k) = D C(X(k + pi), Y(k)) 

1+1 i = 0 

attains a minimum. 

If C is a continuously differentiable function 
defined on an open (possibly infinite) domain in the xy- 
plane, then partial di fferentation of F(k) with respect to 
Y(k) and setting equal to zero yields the necessary 
condition 


T 

E (X(k + pi), Y(k)) = 0 , 

i = 0 

where C 2 (x.y) = r^C(x,y). 

^ 3y 

For example, if C(x,y) = (x - y)^ , then the least 

square criterion leads to 

21 (-2) [x(k+pi)-Y(k)l = 0. 

i = 0 

That is, 

T 

2 ^ X(k + pi)-(r + ]'iy(i) 0. 

i = 0 


A-- 


That is, 
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T 

Y(k) = zr^-T ^ X(k + pi) . 

T + 1 i = 0 

In other words in this case, Y(k) is just the average of 
the (T + 1) values X(k + pi), i = 0, 1, T. 

If C is defined discretely as a cost matrix, then the 
cost F(k) must be minimized directly by determining for 
which column of the matrix the entries C(X(k + pi), Y(k)) 
have smallest sum for i = 0, 1, 2, T. 

Once Y(k) has been found for each k, the average 
cost, F, for the given period p can be determined by 

P 

^ = (1/p) D F(k) . 

k = 1 

Now the most straightforward criterion for determining 
the least cost periodic sequence (Y) is the average 95% 
confidence cost error bound, E, for the p values Y(k), 
k = 1* 2, ..., p. (This step is necessary; otherwise, the 
least cost periodic sequence will always turn out to be the 
whole sequence of data values. That is, T = 1.) 

The average 95% confidence cost error bound, E, is 
given by 


P 

E * (1/p) Z E(k) , 

k = 1 


whore , 
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E(k) 




Here, t is the appropriate t-distr ibution value for T 
degrees of freedom and V(k) is the sample variance in 

costs given by -j- 

V(k) = (1/T) L fc(X(k + pi), Y(k)) -F(k)l^ 

i = 0 L 

Each possible period p generates a periodic sequence 
(Y) and an average 95 % confidence cost error bound E for 
(Y). That sequence (Y) whose average cost error bound E 
is smallest is deemed the best fit periodic sequence with 
respect to the cost matrix C. 
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IfTITIALIZE VA(0) = 3. EA(P) = 0< P * 1 TO INT (NMAX/2) 
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APPENDIX B 

THE EVOLUTIONARY PROGRAM 


An evolutionary program was written for use in this study. 
Input in addition to the data points of concern consists of the 
error matrix, the number of states in the initial finite state 
machine, the start state of the initial machine, the maximum 
number of states allowed, the number of symbols in the data 
alphabet, the length of the initial history to be used and the 
length of the window over which the machines are exercised. The 
window can have either a fixed length, or it can be the total 
history. The alphabet consists of the integers from one to the 
number of different symbols. 

The program constructs, by random assignment of next state 
and output symbol, a finite-state machine with the initial number 
of states and the initial state. It then exercises the machine 
over the window; and, for those state-input pairs exercised, 
assigns that output which minimizes the error. The resulting 
machine is the initial parent machine. It is then exercised over 
the window, the error score is computed, and the prediction of 
the next data point is made. 

Up to five offspring are constructed and scored over the 
window. If an offspring has a better score than the parent 
machine, it replaces the parent machine and it? prediction of the 
next data point is the accepted prediction. If none of the five 
offspring have a better score, the parent machine is retained. 

The available history is advanced one data point and the 
procedure is repeated. 

In forming an offspring, the program randomly chooses one of 
the following mutations: 1. Change up to five randomly chosen 

next state trans i sti ons . 2. Add a state. 3. Delete a state, 
and 4. Change the start state. In each case, the mutated 
machine is exercised over the window and those state-input Pcirs 
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J 

which are exercised are assigned that output which minimizes the 5 

error. The resulting machine is the offspring machine. As before, 
whether or not the resulting machine is retained as a new parent 
depends upon its score being better than that of its parent. 

For debugging purposes, the program was tested over three i 

different environments, one cyclic with a short cycle, one with a 1 

long cycle, and another which was random. In each a four symbol 
alphabet was used, each had 96 data points with an initial history 
of length 50 and a window of length 50. Each entry in the error 
matrix was the square of the difference between predicted value and 
actual value. For the short cycle, the program aft»r four to six 
mutations achieved a perfect score and made all predictions cor- 
rectly. For the long cycle case the error score improved frrv!i 
.796 to .82 in one run and from ,632 to .82 in another. For the 
random environment, the error score fluctuated between 1.06 and 
,449 in one run and between .816 and .612 in another. 

A listing of the program and flowcharts follows. 



CjO 000 
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LISTING OF THE EVOLUTIONARY PROGRAM 


?? p999 

PROGRAM rSMdNPUT, OUTPUT, TAPE1 .TAPE3,TAPE6) 


THIS IS MAIN DRIVER PROGRAM 


C0MMON/IODEF/IMPFSM.INPDAT,lOUrSM 

DIMENSION IFSM(60), MFSM(60), IHS(200), lERMTXOO) 


SET INPUT OUTPUT TAPES 
INPFSM = 1 
INPDAT = 3 
lOUFSM = 6 
RtWIND INPFSM 
RENIND INPDAT 
REWIND lOUFSM 

initial PARAMETERS, ERROR MATRIX AND DATA 

LHLL INRD 

INITIALIZE FINITE STATE MACHINE (FSM) 

CALL INIKNIN, IMCSN.IFSM) 

INITIALIZED MUTATE ROUTINE 


NHSS-NHS 


INFL = 0 
MWSS = 1 
MCSN = • 

CAL L MUT A T ( MF SM , 1 NFL . MWSS , MCSN ) 


IWSS = ISS 


NPRR = 0 
NF'RD - 0 
INFL « 1 
KPRED 


C SET OUTPUTS OPTIMALLY 


lOE 


100 IF (KPRED.EO.l) GO TO 10S 

SSkr ST0PT(LWIN,NHS, IWSS,NIN, IFSM, IHS, lERMTX) 
NnAC = 0 

CALL SCOR(IFSM,IWSS.SCORI,IPRED.IPERR) 

PRINTS. "SCORI= ".SCORI," IPRED- MF'RED," IPCRR= 
PRiNT»,"IWSS = ".IWSS,*' IMCSN = MMCSN 
IF(SCORI.FO.O) GO 10 130 
SET UP FOR Off SPRING MACHINE 
M2 - 2» IMCSN '•MIN 


, IPERR 


DO 112 Kd,S 
DO 110 I d , M2 

MESMO) = irSM(I) 
110 CONTINUE 
MU TAIL PARENl MACHINE 
MWSS - IWSS 
MCSN_:iJMCSN 


D - 


10 rORM OFFSPRING 


OF POO*'l 


CALL MUTAT(MFSM,INFL,HMSS.rtCSN) 

SET OUTPUTS OF OFFSPRING OPTIMALLY 

CALL ST0PT(LWIN»NHS.MWSS,NIN,MFSM,IHS»1ERMTX) 

CALL SCOR ( MFSM , MWSS , SCORM , MPRED , MPERR ) 

IF(SCORM.LT.CCORl) GO TO 115 
112 CONTINUE 

115 PR I NT “SCORM- ", SCORM,' MPRED- “, MPRED, “ MPERR- “, MPERR 
PRINT*, "MWSS - “,MMSS,“ MCSN - ",MCSN 
DOES OFFSPRING HAVE BETTER SCORE 
IF(SCORI.LE. SCORM) GO TO 130 
OFFSPRING HAS BETTER SCORE SO REPLACE PARENT BY IT 
I MCSN - MCSN 
IWSS - MWSS 
M2 - 2*IMCSN*NIN 
DO 120 1-1, M2 

IFSM(I) - MFSM(l) 

120 CONTINUE 
NMAC - 1 

130 CALL UPDATE (IFSM. IWSS) 

UP COUNTER ON CORRECT PREDICTIONS 
KPERR=IPERR 
KPRED = IPRED 
IF(NMAC.EQ.O) GO TO 118 
KPERR' MPERR 
KPRED=MPRED 
118 NPRR-NPRR+KPERR 
NPRD = NPRD+KPRED 
IF(NHS.LT.NDP) GO TO 100 

PRINT*, "NBR OF CORRECT PREDICTIONS - ",NPRD 
APRR-NPRR 

APRR=APRR/(NDP-NHSS) 

PRINT*, "AVERAGE PREDICTION ERROR- ",APRR 
PRINT 900, (IFSM(I),I-1 ,M2) 

900 FORMAT (/,8I4) 

STOP 

END 

SUBROUTINE INIKNINT, JCSN, JFSM) 

% 

: SUBROUTINE SETS UP INITIAL FINITE STATE MACHINE 

DIMENSION JFSM(60) 

DO 100 J-1.JCSN 
DO 100 I-1,NINT 

: COMPUTE POSITION IN FSM TABLE 

NCS - 2*((J*1)*NINT+I) 

: RANDOMLY SELECT NEXT STATE AND STORE IN FSM TABLE 

X - RANF(N^ 

FNS - JCSN*X+1. 

INS - FNS 

JFSM ( NCS- 1 ' • INS 

: RANDOMLY SELECT OUTPUT AND STORE IN FSM TABLE 

X • RANF(N> 

FOT - NINT*X+1. 

B-4 lOT - FOT 

JFSM (NCS) - lOT 
100 CONTINUE 
RETURN 
END 
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SUBROUTINE INRD 

SUBROUTINE READS IN DATA FOR INITIAL MACHINE, 

ERROR MATRIX AND DATA TO BE OPERATED ON 

COMMON/IODEE/INPFSM, INPDAl , lOUFSM 

COMMON/F SMPA/ lERMTX , IHS , IMCSN, ISS , LW!N . MXNS , NDP , NHS . NIN 
DIMENSION lERMTXOO), IHS(200) 

READ DATA FOR INITIAL FSM 

READ ( INPFSM , ♦ ) IMCSN , ISS , LWIN, MXNS , NHS , NIN 

READ IN ERROR MATRIX 
KK = NIN^»NIN 

READ(INPFSM,«)(IERMTX(I),I=1,KK) 

READ IN DATA POINTS 
I = 1 

100 READ( INPDAT.») IHS(I) 

IFdHSd) .LT.O) GO TO 110 
I = I + l 
GO TO 100 
110 NDP ' I-l 
RETURN 
END 


Of POOR QUALITY 


SUBROUTINE MUTAT ( JFSH, INFLT ,JWSS,JCSN) 

THIS SUBROUTINE CHANGES NEXT STATE ASSIGNMENTS, ADDS STATE, 
AND CHANGES START STATE-ALL RANDOMLY 


COMMON/FSMPA/IERMTX , IHS , IMCSN , ISS , LWIN , MXNS.NDP ,NHS, NIN 
DIMENSION JFSM(BO), lERMTXOO), IHS(200) 

IFdNFLT.EO. 1 ) GO TO 100 
: INITIALIZE 

MUCNT = 0 
RETURN 

NORMAL ENTRY 
100 MUCNT » MUCNT-H 
JCSN - IMCSN 

: CHECK IF TIME FOR POSSBILY ADDING OR DELETING A STATE 

: OTHERWISE GO TO CHANGE NEXT STATE ASSIGNMENT 

IF(MUCNT.LT. 10) GO TO ISO 
POSSIBLY ADD STATE 


MUCNT = 0 
X = RANF(N) 

IF(X.GT.O.S) GO TO 200 

ADD STATE UNLESS MAXIMUM NBR OF STATES ALREADY REACHED 
IF( IMCSN. GE.MXNS) GO TO 200 
ADD STATE 

10S JCSN = IMCSN+1 ^ 

RANDOMLY ASSIGN NEXT STATE AND OUTPUT FOR cACH STATE/ 


INPUT PAIR IN NEW STATE 
DO 110 1=1 , NIN 
X = RANF(N) 

FSN = JCSN»X+1. 

NNS = FNS 


NCS = 2^K ( JCSN-1 )*NIN+I) 


JFSM(NCS-I) = NNS 
X = RANF(N) 

EOT = NIN^»X+1 . 

NOT = FOT 
JFSM(NCS) = NOT 

'^RANDOMLY^CHANGE THE NEXT STATE TO NEW STATE FOR FROM 1 TO NIN 
INPUT/STATE PAIRS FROM ORIGINAL MACHINE 
X = RANF(N) 

FK = NIN«X+1. 

KK = FK 

MM = (JCSN-1)«NIN 
DO 120 1=1 ,KK 
X = RANF(N) 

FSI - 2.»(MM«X*1.) 

NSI - FSI 

JFSM(NSI-l) - JCSN 
120 CONTINUE 
RETURN 
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c 

c 


L 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


SET 


OF STATE/INPUT PAIRS 


OF POOR QUALITY 

y^^ETHER TO CAaHQE STATE OK NEXT STATE FOR 
150^X^?”rANMN)^*^ STATE/INPUT PAIRS 

IF(X.GT.O.t) GO TO 170 
randomly change START STATE 
X = RANF(N) 

FWSS “ JCSN*X+1. 

JWSS = FWSS 
RETURN 

CHANGE NEXT STATE RANDOMLY TOR RANDOM 
170 MM « JCSN*NIN 
X = RANF(N) 

FN = 5.*X+1. 

NN = FN 
DO 180 1=1, NN 
X = RANF(N) 

FNS = JCSN»X+1. 

NNS = FNS 
X = RANF(N) 

FCS = 2.^(MM^X+1 . ) 

NCS = FCS 
JFSM(NCS-l) = NNS 
180 CONTINUE 
RETURN 

niiroiJicr^^K STATE IF MORE THAN ONE STATE, 

OTHERWISE GO TO ADD STATE 

200 IF(JCSN.EO. 1 ) GO TO 10S 

DELETES A STATE FROM THE FINITE STATE 
MACHINE, HOWEVER, THE STARf STATE IS NEVER DELETED 

JCSN - NUMBER OF STTES IN FINITE STIE MACHINE 
JFSM - TABLE CONTAINING FINITE STATE MACHINE 
NIN - NUMBER OF INPUT ALPHABET SYMBOLS 

KrISS ■ TO BE DELETED (DETERMINED IN ROUTINE) 

NSTBM - NUMBER OF STATES TO BE MOVED 
NWTBM - NUMBER OF WORDS 10 BE MOVED 
IWFRM - FROM POSITION MINUS ONE 
IWTOO - TO POSITION MINUS ONE 
JWSS - INITIAL START STATE 

DECIDE WHICH STATE TO DELETE 
210 X = RANF(N) 

F = X*JCSN4l.O 
NSTBD = F 

START STATE IS NOT DFl ETFD 
IT (NSTBD. EO. JWSS) GO TO 210 
IMNSTBD.EO. JCSN) GO TO 270 
CAI CULATE NllMBTR OF WORDS 10 MOVE 
NSTBM = JCSN NSTBD 


IN (ABIE MTSM 


NWTBM 


■'•NIN^NSTBM 
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L MOVE ST^TCS TO CLOSE GfiH BUT TIRST GET ADDRESSES 
IMTOO » 2»N1N«(NSTBD-1) 

IWFRM * IMT0042*NIN 
DO 260 I-1,NWTBM 

JFSM(IWTOO+I) = JFSM(IWFRM+I) 

260 CONTINUE 

C DECREMENT NUMBER OF STATES 
270 JCSN = JCSN-1 

C GET SET TO TEST ALL NEXT STATE REFERENCES 
ITEST = JCSN#NIN*2 

C TEST WHETHER OR NOT TO CHANGE NEXT STATE REFERENCE 
DO 290 1=1, ITEST, 2 

IF(JFSMd).LT.NSTBD) GO TO 290 
IF(JFSMd).GT.NSTBD) GO TO 280 
X = RANF(N) 

F = X*JCSN+t.O 
II = F 

JFSMd) = II 
GO TO 290 

280 JFSMd) = JFSMd )-t 
290 CONTINUE 

C CORRECT START STATE, IF NECESSARY 
IF (JWSS.LT.NSTBD) GO TO 300 
JWSS - JWSS-1 
300 CONTINUE 
RETURN 
END 


t .» c i o 


OF POOR QUAUTY 


SUBROUTINE SCOR ( JFSH , JWSS , SCORX , JPRED , JPERR ) 

THIS SUBROUTINE SCORES THE MACHINE OVER THE HISTORY 
IN THE WINDOW 


COHMON/FSMPA/IERMTX,IHS,IMCSN,ISS.LWIN.MXNS,NDP,NHS,NIN 
DIMENSION IHS(200) , JFSM(60) , lERMTXOO) 

SCORX ' 0 
ISN = JWSS 
KK = 1 
ANP = NH5-1 

IS WINDOW TO BE HISTORY*? 

IF(LWIN.LT.O) GO TO 100 
WINDOW IS FIXED LENGTH. 

IF(LWIN.GE.NHS) GO TO 1 
SET UP FOR FIXED LENGTH WINDOW 
KK = NHS-LWIN+1 
ANP = LWIN-1 
SCORE OVER WINDOW 
100 NHSl = NHS-1 


IS IT LONGER THAN HISTORY? 
00 


DO 120 I=KK,NHS1 
IIN = IHS(I) 

NCS = 2*((ISN-1 )*NIN4]IN) 

INS = JFSM(NCS-1 ) 
lOM = JFSM(NCS) 
lOD = IHS(I+1 ) 

: GET ERROR VALUE FROM ERROR MATRIX 

NCS = ( ( IOM-1 )*NIN+IOD) 

SCORX = SCORX+IERMTX(NCS) 

ISN = INS 
120 CONTINUE 

SCORX = SCORX/ANP 
: CHECK IF PREDICTION IS CORRECT 

IIN = IHS(NHS) 

NCS = 2»( ( ISN-1 )*NIN+IIN) 
lOM = JFSM(NCS) 
lOD = IH5(NHS+1 ) 

JPRED = 0 

IFdOM.EO. lOD) JF’RED=1 
NCS=( ( lOM- 1 )^*NIN+IOD) 

JPERR= lERMTX(NCS) 

RETURN 

END 
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c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

L 


SUIR'OUTINE STOPTdWINT.NHSl ,IWSST,NINT,IFSMT,IHST,IERHT) 

LWINT - LENGTH OF THE WINDOW 

NHST ~ CURRENT POSITION IN THE HISTORY (IHS IN COMMON) 

IWSST - WINDOW START STATE 
NINT - ALPHABET SYMBOL SIZE 

IFSMT - FINITE STATE MACHINE FOR SETTING OUTPUT 

lERMT - ERROR MATRIX 

IHST - ADORES OF HISTORY DATA 

ISIPR(I,J) - STATE -INPUT TABLE WHERE 

I IS THE POSITION IN THE TABLE 
J IS 1 , 2 OR 3 WHERE 

(1,1) CONTAINS THE STATE NUMBER 
(I,2> CONTAINS THE INPUT NUMBER 
(1,3) IS TOTAL NUMBER OF OUTPUTS ASSOCIATED 
WITH THIS STATE-INPUT PAIR 

lACOP(I) - THE MACHINE OUTPUT-DESIRED OUTPUT TABLE WHERE 

I IS THE POSTION IN THE TABLE 

ISIPK - THE COUNT OF UNIQUE STATE- INPUT PAIRS IN TABLE ISIPR 

lAOPK - THE COUNT OF TOTAL ENTRIES IN TABLE lACOP 

DIMENSION ISIPR(2S.3).IACOP(SO) .IHST (200) , IFSMT (60), IERMT( 30) , 

1 lERR(IO) 

CACULATE THE FIRST INPUT POSH ION OF THE WINDOW 
NHSWP = NHST-LWINT+1 
SET LOOP TO LENGTH OF WINDOW MINUS TWO 
ILOOP = LWINT-2 
ZERO STATE-INPUT PAIR COUNT 
ISIPK = 0 
ZERO OUTPUT COUNT 
lAOPK = 0 

SET ISNT TO WINDOW START STATE 
ISNT = IWSST 

NOW BUILD STATE- INPUT -OUTPUT TABLES 

FIRST STATE-INPUI-OUIPUT ENTRY HANDLED AS A SPECIAL CASE 
ISIPR(M) = ISNT 
TINT = IHST (NHSWP) 

ISIPR( t ,2) = IINT 

SET OUTPUT COUNT FOR TFIE STAIE- INPUT PAIR TO ONE 
ISIPR( 1,3) = 1 
GET NEXT STATE 

II = 2^( ( ISNT 1 HNINTHINT) 

ISNT = IFSMT(II-I) 

INCREMENT INPUT POINTER 
NHSWP = NHSWP+1 
GET NEXT INPUT 

IINT = IHST (NHSWP) 

STORE DESIRED OUTPUT 
lACOPM) - IINT 
SET OUTPUT COUNT TO ONE 
lAOPK - 1 

SET COUNl OF UNQIUF SfAlE- lNf>UT PAIRS TO UNE 
ISIPK = 1 


B-10 


i 


crr:^:r;;:L u 

CP F?r-C Q JAuiTY 


OF IHF STATE- INPUT -OUTPUT TABLES 


1 


NOW BUILD THE BALANCE 
IS^THIS^STATE-INPUT PAIR IN THE TABLE? 

^°if(ISnt*?ne!isi*pr(ijk,i)) go to too 

IN IHE TABLE 

IJKS = UK 
GO TO 200 

^APPEND NEW STATE-INPUT-OUTPUT TO TABLE 
INCREMENT COUNT 


110 


GET 


I5IPK = ISIPK+1 
ISIPRdSIPK, 1 ) = ISNT 
ISIPR(ISIPK,2) = TINT 
ISIPR(ISIPK,3) = 1 
NEXT STATE ^ 

II = 2^^( (I5NT-1 )*NlNTtlINT) 

ISNT 


L 

C 

c 

c 


IFSMTdI-l ) 

INCREMENT INPUT POINTER 
NHSWP = NHSWP+1 
GET NEXT INPUT 

IINT - IHST(NHSNP) 

INCREMENT OUTPUT COUNT 
lAOPK = IAOPK+1 
STORE DESIRED OUTPUT 

lACOPdAC^K) = TINT 

CHECK°ir°THE IDENTIFIED STATE-INPUT 
200 lEdJKS.NE.ISlPK) GO TO 300 
ARRIVE HERE. STATE-INPUT IDENTIFIED 
GET NEXT STATE AND OUTPUT 

II = 2»t dSNT-1 )*NINT+IINT) 

INCRLMENl’oUlPuVcnUNl for INPUT PAIR 

ISlPRdSIPK,3) = ISIPR(1SIPK.3) + 1 
ICREMENT INPUT POINTER 


PAIR IS LAST IN THE TABLE 
IS LAST IN THE TABLE 


GET 


'nHSWP = NHSWP+1 


L 

c 


NEXT INPUT 
IINT = IHST(NHSWP) 
INCREMENT OUTPUT COUNT 
lAOPK = IAOPK+1 
STORE DESIRED OUTPUT 

IACOP( lAOPK) = IINT 
GO TO SOO 

ARRIVE HERE. NECESEfARY 10 


Cfx'EATE SPACE IN TABLE FOR OUTPUT 


PACE SHOULD BE IN TABLE lACOF 


300 


DETERMINE WHERE 
ISUM = 0 

DO 320 111=1 . IJKS 

ISUM - ISUM+ISIPRl 111.3) 

T20 CONTINUE , 

.MOVE lACUP FNlRlfS 10 HAKI ‘ '‘(K.E 
I MOVE = lAOPK I!. UM 
INLM = lAOPK+2 
101 D = lAOPKd 


br.'G*?3 Vi- r.^r” 

OF POO^J O’.I/Ui I 


DO 330 1-1 ,It107E 

lACOP(INEW-I) - lACOP(IOLD-I) 

330 CONTINUE 
GET NEXT STATE 

II - 2*( (ISNT-1 )^»NINT + IINT) 

ISNT = lESMT(II-l) 

INCREMENT INPUT POINTER TO GET NEXT INPUT 
NHSWP - NHSWP+1 
IINT - IHST(NUSMP) 

STORE DESIRED OUTPUT 

1AC0P( lSUM+1 ) - IINT 
INCREMENT OUTPUT COUNT 
lAOPK = IAOPK-»1 

INCRfMENT OUTPUT COUNT FOR THIS STATE- INPUT PAIR 
1S1PR(IJK5,3) - IS1PR(IJKS,3)-M 
SCO CONTINUE 

NOW THAT TABIES iSIPR AND lACOP ARE CONSTRUCTED, MACHINE 
OLIIPUT CAN PE DF TERMINIST ICAl LY SET TO MINIMIZE ERROR 

St T loot’ TO SET OUTPUT FOR THOSE STATE INPUT PAIRS EXERCISED 
GOG DO 800 1= 1 , ISTPK 



ir ( 1 .NE . n 

GO TO E>OA 


lACKN -0 



GO 10 608 


G04 

IACKN-IACKN+ 1S1PR( 1-1 ,3) 

608 

DO 610 L=1 

.NINT 


I T RR ( L ) 

- 0 

610 

CONTINUE 



SET HOOP TO NUMF'.EK OF TIMES THIS STATE- INPUT PAIR EXERCISED 
H OOP - 1SIPR(1,3) 

SET LOOP TO TRY AU THE Al TFIAPt T 
DO 700 J-1 .NINT 
no 700 1 . ILOOP 

CAT cm ATE (OSITION IN ERROR MATRIX TO OBTAIN ERTTOR FOR 
THIS TENTATIVE OUTF^Ul AND THE EXPERItNCES OUlPUT 
M - (J 1 ) »NINT4lAC0P(K-ilACKN) 
lERR(J) - 1ERR( J)* lERMT(M) 

700 CONTINUE 

NOW FIND TTTF OUTT*UT I'RODUCING THE LEAST ERROR AND SELECT IT 
I COMP - ITRR(I) 

ST T TSOP • I (TTNTAllVE OUIT^UT) 

IS.OP = 1 

DO 7S0 IJ-P.NINT 

IF ( IT RR( TJ) .GT . TCOMP) GO TO 7S0 
ICOMP - ITRR(IJ) 

TSOP - IJ 
7S0 (ONTINUE 

( AT dll AIT WHTRT Dl liRHTNFD OUlPUT IS TO HT STORED 
ISO ?» ( ( i;.|PK( 1 . 1 ) 1 ) ’•NlNl ♦ ISTPKH 1 ,2) ) 

SI T [11 Ti RMINTSI IC OOIPUI 
IT SUM ISO) 15. OP 

nin ('('NTiNiiE 
Rl TURN 
END 
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OF POOR QUALITY 


SUBROUT INL UPDATE (JFSM, JMSS) 

THIS SLIE5R0UTINE UPDATES START STATE. CURRENT HISTORY 
LENGTH 

COMMON/LSMPA/IERIITX.IHS. IMCSN, ISS.LWIN.MXNS.NDP.NHS^NIN 
DIMENSION JFSM(60), IHS(200), ILRMTXOO) 

C UPDATE HISTORY LENGTH 
NHS * NHS+1 

C IS WINDOW EQUAL HISTORY LENGTH 
IF(LWIN.LT.O) GO TO 100 
C IS WINDOW LONGER THAN HISTORY 
IF(LWIN.GE.NHS) GO TO 100 
C SET UP LOR FIXED WINDOW 
KK ^ NHS-LWIN 
IIN = IHS(KK) 

ISN = JWSS 

C UPDATE START STATE 

NCS - ?*( ( ISN-1 ) <»NIN+1IN) 

INS - JFSH(NCS-I) 

JWSS = INS 
100 RETURN 
END 

LND Of F ILE 


MAIN DRIVER PROGRAM 


READ IN 

PARAMETERS, ERROR 
MATRIX AND DATA. 
CREATES INITIAL 
FINITE STATE 
MACHINES 


CALL STOPT SUBROUTINE' 
OPTIMIZES OUTPUT FROM 
, FSM OVER WINDOW , 


MAS 

PREDICTION 

^^ORRECT^ 


CALL SCOR SUBROUTINE' 
SCORES PARENT MACHINE 
. OVER WINDOW j 


PERFECT 


CALL STOPT OPTIMIZE 
OUTPUTS FROM 
OFFSPRING OVER 
WINDOW 


CALL SCOR SCORES 
OFFSPRING MACHINE 
OVER WINDOW 


CALL MUTAT CREATES 
OFFSPRING MACHINE 



^^^^spring'\ 

SCORE BETTER THAN 
E NT^^^ 


REPLACE PARENT 
BY OFFSPRING 


^^HAVE 

ENOUGH 

.OFFSPRING BEEN 
^v^^RIEO^-^ 


CALL UPDATE; UPDATES 
HISTORY AND WINDOW 
START STATE , 


IS ' 
DATA 

FINISHED 
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ki_liLLTS AvD 





















SUBROUTINE MUTAT 


OF POOR QUALITY 


SET COUNT 
TO ZERO 



IS IT 

INITIALIZED 


RETURN 


INCREMENT COUNT 


RESET COUNT 
TO ZERO 


COUNT EQUAL 


CHOICE TO ADD 
^^\^TATE^ 


RANDOMLY CHANGE 
SrART STATE 


.x^HOICE TO^ 
CHANGE START 
STATE 


HAS 

MAXIMUM NBR 
STATES BEEN 
Annrn ^ 


RANDOMLY CHOOSE 
FROM ONE TO FIVE 
STATE-INPUT PAIRS 


RANDOMLY CHOOSE 
A NEXT STATE FOR 
EACH OF THESE 
STATE-INPUT PAIRS 


STORE IN 
CORRESPONOING 
POSITION IN 
FSM TABLE 


UP CURRENT 

STATE 

COUNT 

AND RANDOMLY 

ASSIGN NEXT 

STATE 

AND 

OUTPUT 

FOR 

EACH 

STATE- 

INPUT 

PAIR 

IN NEW 

STATE 


CHOOSE 

A NBR 

FROM 

ONE TO 

NBR OF 

INPUT 

SYMBOLS 

. RANDOMLY 

SELECT 

THAT 

MANY 

STATE- 

INPUT 

PAIRS 

1 ^ROM THE ORIGINAL 

MACHINE AND 

FOR 

THESE 

SFT NEXT 

STATE TO NEW 

STATE 1 


RETURN 
















DELETE A STATE 


SELECTED ^ 
STATE • START 
STATE ^ 


START STATE < 
s^ATE DELETED^ 


DECREMENT START 
STATE BY ONE 


CALCULATE NUMBER 
WORDS TO BE MOVED 
TO FILL UP GAP 


CALCULATE ADDRESS 
AND MOVE STATES 


DECREMENT NUMBER 
OF STATES BY ONE 


SET TO TEST 
ALL NEXT 

STATE REFERENCES 


-^NEXT STATE^ 
REFERENCE < 
sJTATE DELETED^ 


^NEXT STATE 

reference > 
sw state OELETEC 


SET NEXT STATE 
REFERENCE RANDOMLY 
TO ONE OF 
I THE STATES 


oecrement next 

STATE REFERENCE 
BY ONE 


FINISHED? 
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SUBROUTINE SCOR 


SET START OF 
WINDOW TO ONE AND 
SCORING COUNT TO 
CURRENT HISTORY 
COUNT MINUS ONE 


' IS WINDOW > 
TO BE CURRENT 
Vv HISTORY / 


X IS \ 
WINDOW 

LENGTH FIXEO^ 
BUT LONGER THAN 
S. CURRENT X 
\ HISTORY X 


SET START OF 
WINDOW TO CURRENT 
HISTORY COUNT MINUS 
WINDOW LENGTH PLUS 
ONE AND SCORING 
COUNT TO WINDOW 
LENGTH MINUS ONE 


SET END OF WINDOW 
FOR SCORING TO 
CURRENT HISTORY 
COUNT MINUS ONE 


FOR EACH POINT IN 
WINDOW GET MACHINE 
OUTPUT AND DESIRED 
OUTPUT. FROM THESE 
GET ERROR VALUE 
FROM ERROR MATRIX 


SCORE IS SUM OF 
ERROR VALUES 

DIVIDED BY 
SCORING COUNT 


IF PREDICTION AT 
END OF CURRENT 
HISTORY IS CORRECT, 
SET JPHCO - 1. 
ELSE ZERO. ALSO 
RETURN ERROR 
VALUE 








OF 
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STOPT-2 


IDENTIFIED 
STATE-INPUT 
PAIR LAST IN 
. THE TABLE 


DETERMINE WHERE SPACE 
FOR DESIRED OUTPUT 
SHOULD BE CREATED IN 
TABLE lACOP; CREATE 
SPACE IN TABLE BY 
MOVING ENTRIES BELOW 


GET NEXT STATE; 
INCREMENT OUTPUT 
COUNT ASSOCIATED 
WITH THIS STATE- 
INPUT PAIR; GET NEXT 
INPUT; INCREMENT 
OUTPUT COUNT; STORE 
AS A DESIRED OUTPUT 
FOR THIS STATE- 
INPUT PAIR 


GET NEXT STATE; GET 
NEXT INPUT; STORE AS 
A DESIRED OUTPUT; 
INCREMENT OUTPUT 
COUNT; INCREMENT 
COUNT FOR THIS 
STATE-INPUT PAIR 


FINISHED 


NOW THAT 
TABLES 
ISIPR AND 
lACOP, 
STATE- 
INPUT AND 
OUTPUT 
TABLES. 
OUTPUT CAN 
BE SET DE- 
TERMINIS- 
TICALLY TO 
MINIMIZE 
ERROR 


SET lACKN . 0 SO THAT 
lACKN WILL BE ZERO 
FIRST TIME THROUGH 
I LOOP TO COME 
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CONTROL OF ROBOTIC DEVICES 
6^ George H. Burgin, Decision Science, Inc. 


SUMMARY 


The control of robotic devices Is a challenging and important 
task. Two primary factors contribute to that challenge. 1. The 
differential equations describing the behavior of a multilink 
manipulator turn out to be very complex and tedious to derive. 

2. The manipulator is a highly non-linear device so that conven- 
tioanl control system design techniques are applicable only to a 
1 i mi ted extent . 

The presently applied or proposed control techniques are 
reviewed, particularly the "inverse problem" technique, the com- 
puted torque technique and the model -ref erenced adaptive control 
technique, and a new technique, which uses finite state machine 
gain adjusters, is proposed. 

Next, the differential equations for a representative two- 
link model (upper arm and a lower arm) are derived and a basic 
linear feedback control system, operating about some linearized 
position of the system is designed. The entire system is simulated 
on a digital computer and representative time histories of responses 
to step inputs are shown, first for a control system which uses 
fixed gains over the entire operating envelope. 

Next, a finite state machine (FSM) gain adjuster for the 
shoulder joint is designed. It receives an 8-symbol input 
alphabet, which encodes information about error and error rate. 

The outputs of the finite-state machine are the gains for the 
proportional and rate feedback signals. 


A few representative sample reponses show the superiority of 
this control system over the one with fixed gains. 

It is recommended to implement such a control system on the 
actual Puma hardwave and to compare its performance with the one 
predicted in this report. 
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INTRODUCTION 


For this discussion, we will limit ourselves to mechanical 
manipulators which represent a subset of robotic devices. We 
exclude, therefore, such devices which might be used to explore-- 
remotely controlled or autonomously-- such things as surfaces of 
planets. 

Mechanical manipulators are chains of linkages connected by 
joints. Joints may be rotational or translational. Without loss 
of generality, we will assume that each joint has one degree of 
freedom; multiple rotational degrees of freedom can be represented 
by links of zero masses and length, each one with one single degree 
of freedom. 

A typical industrial manipulator has six joints, seven links 
and a gripper (also called endef fectors ) . Figure 1 shows a com- 
mercially available manipulator, the PUMA 250, manufactured by 
Unimation, Inc. Two such manipulators are in operation for 
research purposes at the NASA Langley Research Center. Table I 
summarizes some of the key specifications of this manipulator. 

Mechanical manipulators of this type have become increasingly 
important in recent years, and a great deal of effort has been 
spent for research in the area of simulation and control of 
industrial manipulators. During the 1981 joint automatic control 
conference in Chari ottsvi 1 1 e , not less than eighteen papers were 
presented on this subject! (l) 

SYMBOLS 


A,B,C,0 

g 

J 


geometrical constants of system 
acceleration due to earth gravity (9,81 m/sec ) 
moment of inertia of upper arm about its center of mac' 
inertia matrix , 


Kj gain of integral feedback path 

Kp gain In forward loop 

Kp gain of proportional feedback path 

gain of rate-feedback rate 

distance between shoulder joint and elbow joint 

nip mass of payload 

mass of upper arm 

m2 mass of lower arm 

q. i-th generalized coordinate 

i-th generalized force (moment) 

rj distance from shoulder joint to center of mass of 

upper arm 

r2 distance between elbow joint and center of mass of 

lower arm 

RS magnitude of step-input at elbow joint 

RSj magnitude of step-input at shoulder joint 

s Laplace operator 

T kinetic energy 

V potential energy 

ci.B.Y.'S.e system variables which are functions of 

C damping factor 

commanded angle 
^c 

commanded angle 4»2 
‘^c 

T vector of generalized input forces (moments) 

natural frequency 
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Threshold value for absolute value in error of 
for encoding for input symbol to FSM 
Threshold value for error rate for encoding for 
input symbol to 3M. 
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ROBOT CONTROL BACKGROUND 


Equations of Motion 

Mechanical manipulators are highly nonlinear devices. The 
primary cause for the nonlinear nature of manipulators is the 
changing moments of inertia of the various links. In the expres- 
sions for the moments of inertia, trigonometric functions of the 
generalized coordinates appear. Since the generalized coor- 
dinates vary over a wide range (sometimes over a full 360°), 
linearization of the trigonometric functions cannot be performed 
over the full operating range of the manipulator. Other important 
nonlinear terms are products of derivatives of generalized coor- 
dinates with trigonometric functions of generalized coordinates. 

!n addition to these nonlinear effects, which are due to the 
changing physical configuration of the manipulator as it moves 
through space, there are the usual nonlinearities associated with 
any device with moving parts linked by joints: nonlinear friction 

effects, hysteresis, and so forth. 

Much of the literature on robotic manipulators is concerned 
with the formulation of the equations of motion. This problem is 
by no means trivial. Walker and Orin (2) point out that for 
mechanisms with only two or three degrees-of-freedom , these 
equations can usually be derived manually but that for mechanisms 
with more than three degrees-of-freedom, a separate computer 
program is required to symbolically derive the equations of motion. 


The Different Control Schemes 

In the available literature, a number of schemes to control 
mechanical manipulators have been proposed. 

The lnverseJ>_rob;[em Technique: Here, the reojired irput to-quc 

for each joint is computed as a function of t ne desired joint 
acceleration, q^, the joint velocity and tne joint position 
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and the actual q, q and q. To explain this method, consider the 
general form of the equations of motion for a six joint manipulator 
to be: 

J(q)q + Vq + f(q,q) + g(q) = t (1) 

where J(q) is a 6 by 6 inertia matrix 

y is a 6 by 6 viscuous friction matrix 

f(q,q) is a 6 by 1 vector defining Coriolis and 

centrifugal force terms 

g(q) is a 6 by 1 vector defining the gravity forces 

T i s a 6 by 1 vector of the generalized input forces 

(moments ) 

Then, the desired input torque vector is computed as: 

T = * Kj(q^-q) + K 2 (q^-q) 

3. Ic 

where Kj and K 2 are some gains. (Luh, et al in (3) assume and K 2 
as being seal ar gain constants. One of the purposes of this inves- 
tigation is to determine whether the manipulator performance could be 
substantially improved by replacing these scalar gain constants by 
automatically adjusted [by finite state machines]) gain vectors. 

Ideally, we would like to have q approaching q^, then we 
would have 

u = s 

fc(Q.p) = f(q.q) 

9c (q) " 9v_q) 

*The subscript c indicates values computed y the contjol 



program . 


1 
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If these four conditions were satisfied, the equation could be 
written as 

T = J(q)|q^+Kj(qj-q)+K 2 (q^j-q)| +Vq +f (q ,q ) +g (q ) (3) 

Now, we can equate (1) and (3) and we obtain 

J(q)[qji-q+Ki(qd"<i)+*^2^^d‘‘’^} ' ° 

If we call q-q^j the position error e^ , and keep in mind that the 
intertia matrix J(q) is nonsingular, we obtain 


+ K,e„ + K,e^ = 0 
q 1 q 2 q 

This then leads to a control system of the form as shown in Figure 2. 


This method appears to be restricted to those applications 
where the trajectory of the hand is preplanned, which makes it 
possible to know exactly, all the way along tha manipulator's path, 
q, q and q (In other words, these quantities become 
q^, q^ and q^ in the above equations.) 

Reference (3) mentions that "proper choice of values for and K 2 

guarantees the convergency of errors. It does not coordinate the 
speed of convergence for all six joints. Thus, some joint may 
converge faster than the others." Reference (3) is very vague on 
the proper choice of and , which seems to be a key problem 

The Computed Torque Method: Earlier work performed in the field of 

manipulator control system design compared the computed torque method 
with conventional position servo control (4). This is a Jet Pro- 
pulsion Laboratory internal report and could not be made available 
in time for the preparation of this report. Important for the 
computed torque method (and any other method which requires the real- 
time calculation of the generalized moments and forces) is the 
efficiency by which these torques may be computed. This efficiency, 
in turn, depends on how the equations of motion, forming the basis 
for these torque calculations, are formulated. Much of the modern 
literature on robot control is devoted to this problem, reference 
(2) presents a good summary of this subject. At the present time, 
the consensus among researchers in this field seems f.c thct i..< 
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formulation by the Newton-Euler rcLhod yields computationa 1 1 v mc > 
efficient equations than the formulation by Lagrange's method. It 
is usually claimed that in the Newton-Euler approach, the computa- 
tion time grows linearly with the number of links, whereas in the 
Lagrange approach, it grows with the fourth power of the number. of 
links. Therefore, any method relying on calculating the torques 
on-line in real-time, almost has to use Newton-Euler for deriving 
the equations unless the number of links is very small. 


Very recently, however. Silver (5) in a benchmark paper has 
shown that it is possible to overcome some of the difficulties 
generally attributed to the Lagrangian method. He uses a recursive 
Lagrangian formulation such that there is no longer a fundamental 
difference in the computational efficiency between Lagrangian and 
Newton-Euler formulations. 


The Mode l -Referenced Adaptive Control Technique: - This technique 
was primarily developed by Professor Dubowsky at UCLA and was first 
described for the continuous system in (6), and was recently expanded 
to the Discrete-Time case as described in (7). The requirement for 
robots which deliver uniformly high performances over a wide range 
of systems operating conditions precludes the exclusive use of 
classical linear control systems. Adaptive mode 1 - referenced control 
system can "learn" to compensate for nonlinearities arising from the 
various geometrical configurations of the manipulator, and they may 
also be designed to adapt for changing payload characteristics. 

This is especially important for manipulators employed to retrieve 
satellites or parts of satellites of unknown mass. 

Figure 3, reproduced from (7), shows the block diagram of a 
continuous model referenced adaptive control system. It uses a 
linear, second order reference model and the adaptation occurs on 
the gains Kp(i) (positional feedback gain for all joints, that is, 
i = 1...6 for a 6 joint man i pul a tor ) , and the rate feedback gains 
K^(i). An interesting finding of this paper was that for parameters 
of common industrial man i pul a tors , and do not need to be 
varied independently. 
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Gains Adiustpd bv Finite-State Machines_:_ This study proposes a 
new approach to solve the robot control problem. Rather than 
employing an algorithm which adjusts the gains Kp(i) and K^(i) 
based on the observed difference between the robot's response to a 
command input and the reference model's response to the command 
input, the gains will be adjusted based on the output of a finite- 
state machine which receives as input the robot's response. A 
similar approach was used previously to adjust the gains of an air- 
craft stability augmentation control system (8). To derive the 
algorithm for gain ad j ustment . • a specific manipulator was selected. 


DEVELOPMENT OF A ROBOT CONTROL SYSTEM 


Model Definition 

The Puma 250 manipulator was chosen as a device to be controlled 
for two reasons; First, it is a manipulator typically representing 
today's commercially available manipulator, and second, two Puma's 
250 are presently in use at the NASA Langley Research Center. This 
made it possible to obtain physical data about the manipulator which 
are. in general, not available from a manipulator's manufacturer. 

It has been shown, for example in (7). that for most applica- 
tions. the motion of the wrist Joints have little effect on the 
dynamic performance of the lower joints. It is. therefore, justified 
to assume the wrist joints (endeffector joints) of the Puma to be 
locked with respect to the lower arm. 

Another simplifying assumption was made for this study. It is 
assumed that the waist rotation is decoupled from the shoulder and 
elbow rotation. This is the case if the manipulator does not move 
simultaneously around the waist joint, shoulder joint and elbow 
joint, but keeps the shoulder and elbow joint in a locked position 
while it is moving around the waist joint. The study, therefore, 
concentrates on analyzing simultaneous motions aoout shouiuer 
elbow joint. The manipulator model, the-efore. is a ti.o link . ■ i 
two joints system as shown in Finiirc 4. 
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Derivation of the Equations of Motion for the Selected Model 

In the previous section^ Newton~Euler and Laqranqe were compared 
primarily with respect to the comoutational efficiency of the 
resulting equations of motion. It is important to realize that in 
the proposed control system, there is no solution (and not even a 
formulation) of the manipulator's equations of motion required. The 

reason why we need to know the equations of motion and why we 
have to solve them by some numerical method is for the purpose of 
simulating the closed-loop system of manipulator and control system. 
There exists no requirement to perform this simulation in real time. 
Computational efficiency is, therefore, of no practical importance. 
This is quite. in contrast to the "inverse plant" and the "computed 
torque" technique, where torques have to be computed online and in 
real time. We can, therefore, compare the two techniques to obtain 
the equations of motion simply based on their relative merits of 
simplicity of the derivation. 

Newton ' s-Euler* s Method: Applying this method to the system 

shown in Figure 4 results in two translational and one rotational 
equation for each one of the links, resulting in 6 second order dif- 
ferential equations. But, obviously, the system has only two degcees 
of freedom; therefore, we have to formulate kinetic constraint 
equations; for example, the coordinates of the elbow joint in 
inertial space must be the same for the upper arm as for the lower 
arm. It is our experience that considerable skill is required to 
formulate exactly the right number of constraint equations, certainly 
a drawback of the Newton-Euler Method. On the other hand, proper 
formulation in the Newton-Fuler method will provide internal reaction 
forces and moments, presenting an important advantage in robot design. 

L agrangian Method - Once appropriate generalized coordinates 
have been defined, the Lagrange's method is relatively straight 
forward: Express the system's kinetic energy in terms of the 

generalized coordinates and their derivatives, 'et the k’net 
be T. Express the potential eneri;/ (inclutiing all conservative 
forces) as V, then Lagrange's equation states; 

C - ’ • 
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d 

dt 



+ ly 

9q^ 3q. 




i =1 . . . n 

n=number of degrees 
of freedom 


Thus, a system of n second order differential equations will result. 
Note that is the generalized force (or moment) for the i-th 

equation, all nonconservative forces and moments must be included in 
Q. , specifically, the externally applied moments at the individual 
joints. Table II summarizes some of the advantages and disadvantages 
of the two methods. 

Lagrange's Method for Two-Link System: - A natural choice for 
the two generalized coordinates in the two-link system are the two 
angles 4)^ and <*>2 as shown in Figure 4. 

T = 1/24^^ I ^ ^ 

+ 1 (4^+42) 


Letti ng 



2 2 2 
A = Ij + m^r^ + + ”’p*^i 

B = (m2r2 + nip<i2^ 1 

2 2 
C = I 2 m2r2 + nip I 2 

D = + (ni2+mp) 
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and then formulating Lagrange's esoations and collecting ttrii.s y. - is 


Jj(A+C+ 2 B cos 4)2) + <J>2 + B COS42) = 


= <j>2(2(J)^+(J)2)B si n (|)2-Dg COS 


— ^ CO s (<}) + 4 > 2 ) Q 1 


<|»j (C + B cos 4>2) + <t >2 C = 


= -<(),^B sin4>2 ~ ^ cos {<t>j+4)2^ 


It is easy to show that is the applied (control) moment in the 

shoul der joint, which we will call , while Q 2 is the applied 
(control) moment in the elbow joint, let it be Mp. 

To be able to solve the above system of two second order differential 
equations by standard numerical methods, we have to solve them for 
and ip • Before doing this, we introduce the following; 

a = A + C + 2B cos 4>2 


so that 


or 


and 


6 = C + B cos v ^2 
6 = C 

Y = 4 > 2 -Dgcos (jj^-Bg/i^ cos (({(j + dJp) 

e = -Bsin (j)^ ~ cos (4>, + ^Jp) 

^ 8 . 


ct4>i 

4 6(j)p 

- V 4 

64>i 

4 5(J)p 

= e 4 Mp 


-> 

-V 


[A] i 

= b 


[A]’^ 

M 

, 1 
1 


— ^ 


i 

- 


6 -3 i 

a 

1 ’* A 

1 

- 3 


1 



and, solved for 


and 
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4>i 


= |6(y + Mj) - B(c + M2)| 


(j) _ _l/-6(y + M^) + a(c + ^2^1 

2 ” A I f 


This concludes the derivation of the equations of motion. It is 
obvious that any increase in the number of links soon increases the 
complexity of the equations of motion beyond what one can derive 

manually. 


Design of the Basic Linear Feedback Control System 

For a preliminary design of the control system, certain estimates 
about the geometrical dimensions, the masses and the moments of 
inertia of the Puma manipulator had to be made. Figure 5 illustrates 
our assumption about the shape of the lower arm. The distance 
between elbow joint and the waist joint was taken from Unimation's 
drawing of the Puma, as shown in Figure 1, all other geometrical 
dimensions in Figure 5 are estimated values. Note that for a first 

design, we assumed the arm to be homogeneous; this assumption is 

quite inaccurate since in reality, the mass of the arm is concen- 
trated around the two joints (where the DC servo-motors are placed) 

and little mass is around the c°nter of the arm. 

If we designate with A the area of the arm, and with V its 

volume, it follows from Figure 5: 

.15+0.1 p ^2 

^ * 0.575 m = 0.071875 m 

2 

o “33 

V =0.0‘^1875 * 0.03 m"^ = 15625 10 m 

Assuming a homogeneous mass distribution of 7860 kq/m , the aims 
total mass will be 16.95 kq. 
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Under the same assumptions, we may calculate the area's moi'i'" 
of inertia about its cente.' of mass, which is approximately 

2 

For simplicity, we assume the same geometry for the upper arm as we 
have shown in Figure 5 for the lower arm. We can, therefore, sum- 
marize the constants appearing in the equations of motion as follows: 

ei = £2 ^ 0.42 m 
'^1 ~ ~ 

mi = m^ = 16.95 kg 

I = lo = 0-^ 

^zz ^zz 

-2 

g = 9.81 m sec 

m = 2 . 5 ko 
P 

A Fi rst-Cut Design for the Lower Arm Alone : To get a reasonable 

structure of the control system and approximate values of gains such 
that the response of the arm to a commanded step input in angular 
displacement follows a desired second order response type such that 
the physically available control torques of the Puma's DC servo 
motors are not exceeded, we will now proceed to determine values for 
the two gains Kp (gain of the proportional feedback) and Kp (gain 
of the rate feedback, called in Dubowsky's paper (7). 

Figure 6 shows a control system with proportional and rate 
feedback for the lower arm, in which M(iTip) indicates the moment 
about the elbow joint due to the payload. Lineariziiuj ^he i^stei.. 
about ,‘'2 ® yields the following equations: 
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E(s) = R(s) - KpS4>^(s) - Kp(i.2(s) 


I(s) = KpE(s) + M(m ) 


• '(s) 

■2 

elbow 


which results in the following transfer function for 4-2. 


1 


I2 

^^(s) = 


^KpR(s) + M(mp) 


§2 + s + 


I 


1. 


elbow 


e 1 bow 


To obtain reasonable values for Kp, and Kp, we can first neglect 

the gravity term M(mp) and, without loss of generality, we can set 
Kp = 1 so that we can write the transfer function in the standard 
form for linear second order systems. 

K. 


4>2( s) 

R(s) 


•F/, 


I 


e 1 bow 


5 2 . ^ S . _ 


I 


I 


el bow 


el bow 


which we may compare with the normal form; 


such that 


Clii = .V 

R(s) 


s + 2 s * 




and 
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e 1 bow 


^F-»=R = 2-v . 

n 

^ 2 
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We win determine Kp and Kj^ so that, for a step input of R, 
the integral of time multiplied by the absolute value of the error 
(lATEl will be minimized; in other words: 


lATE = / |E(t)| -t dt mi n imum 

It is well known that for a linear second order system lATE is 
minimized if 

C -.'ll- 

2 

(See for example Reference 10, page 93). 

We may specify a second condition which we want to satisfy, for 
example the time to the first peak, which is (reference 9, page 30). 


t 



n 



Substi tuti ng 


~2~ 


for 




we obtain 


n /T 



For the Puma manipulator, a time of 0.5 seconds to the first peak 
appears to be a reasonable, physically relizable choice; thus. 




n 


JL-Z^ ^ g Qg 

0.5 


The moment about the elbow is enuai to: 


I, = + r.mj, = 3.38 kg m 

‘^clbow ^zz 
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Thus, for the linearized system: 


''f • 


elbow 


? 2 -2 
■to) = 266 m kg sec 

* n ^ 


and 

2 ^ 0 ) 1 2 

Kj^ ^ ” ^elbow = 0.1593 sec 

At this point, it seems highly desirable to simulate the actua l , 
nonl inear manipulator system, using the feedback system as developed 
above but dropping the assumption of small angular displacements 
about ^2 ~ doing so, we will gain insight into how far the 

results developed for the linearized system are valid for the actual, 
nonlinear system. We interrupt, therefore, the development of the 
control system at this point and describe the digital computer 
simulation of the manipulator. 


DEVELOPMENT OF A MANIPULATOR COMPUTER SIMULATION 

Overvi ew 

The goal of the computer simulation of the manipulator is to 
have a tocl available which is flexible enough to allow the analyst 
to investigate many different control system designs. It tS, there- 
fore, reguired that the simulation be modular and well structured. 
One of the difficulties in any digital simulation of a continuous 
system is the proper choice of the integration step size; this is 
particularly important in the simulation of a manipulator receiving 
step inputs for angular positions. It is, therefore, reguired that 
an integration method which allows an easy automatic step size 
adjustment, based both on absolute and relative error criteria, be 
used. Remember, we are not simulating a linear second order (or 
higher order) system, but a highly nonlineir syston whoro fH.. -.oi ^ 
zero location continuously chance.', as the i;ia n i pu 1 a t*' • Itiw-. m.).- 
relative to each other. A Run y<‘ ■ • 1 1 1 a type i n t,e gr ♦ ' on ■ I 


most appropriate in a situation like this. Fehlbery (12) dovol- i 
highly accurate fifth order Runge- Kutta method. In one of the most 
recent simulation packages, developed by Pritsker, this method is 
used (13). It seemed appropriate, therefore, to use SLAM as 
simulation language for the control system development. 


The Final Version of the Control System With Fixed Gains 

The Single Arm Control Loop: - Figure 7 shows the final version of 
the control device, using fixed gains. As can be seen by comparing 
Figure 7 v;ith Figure 6, a term proportional to the error integral 
has been added to the control system, so that we have more or less a 
conventional PID regulator. The error- i ntegra 1 term will force, for 
a step input of R(s), the angular displacement of the arm to assume, 
at steady state, the desired angular displacement. 

It is easy to show that the transfer function for Figure 7 can 
be expressed as: 


♦ (s) = 


I 


R(s)Kj 

- + s 


R(s)K^- + M 


zz 


2 2 


s3 + s? 


+ s 


I 


zz 


I p ^ p p 


\ zz 


+ - 


KpK, 


Z Z 


The steady state behavior for a step input R{s) - R is 


lim 4>(t) = lim s 'ii(s) - R 
S"*’0 


J 


7 


f 


i 


Since 


R 


Kp * 1, lim <j>lt) = 

t -► OO 

which is the desired steady state response. 

The Stability of the Sing le Arm Control Loop; - Due to the 
changes in the moments of inertia for large angular deflections, 
it is difficult to determine stability boundaries. However, one 
gets a good "feel" for the stability of the system by performing a 
root locus analysis for the above defined transfer function. This 


requires 

the solution 

of 

the cubic equation. 


. KpK, 

+ 

s ^K,K„ ♦ KpKp'^ + 


'zz 


1 >ZZ / 

or 




s3 

+ a^s^ + 

a2S 

.aj - 0 


which can be shown to be equivalent to finding the eigenvalues of 
the matrix A where 

( -aj -32 - 33 '^ 

1 0 0 

0 1 0 

(See for example ref. 14, page 233) 

A computer program to find eigenvalues was available (EIGEN, 
California State University, CTS). Figure 8 shows a typical root 
locus plot for the above transfer function for variable payload 
masses. It shows the roots to lie in the desirable region for 
payloads from 1 kg to 5 kg and be accept.ble f- 

payload. 


Magnitudes and Signs for Single Arm : - Remember that the analyses 
performed so far were for a system linearized about <l >2 = 0. It 
is therefore necessary to investigate the dynamics for large step 
inputs, because the moment of inertia depends on the angle 
The response of the system will also be asymmetrical for a 
commanded step input of the same magnitude but in opposite 
directions. Tnis is illustrated in Figure 9. The system was 
initially at rest with the upper arm fixed at = 0 and the 

lower arm supported with ^2 ~ 0. At time t = 0^, the support 
was removed and simultaneously the step input command was applied. 
Control system gains were as indicated on Figure 8 which 
guaranteed that control torques did not exceed those physicaly 
attainable on the PUMA (see Table 1). Commanded step inputs were 
^2c~ ^ degrees, $2c~ degrees, and <t2c~ degrees. The 

difference in response to a +135 degree and a - 135 degree 
commanded angle is interesting and can easily be explained by 
noting that in the first case, the moment generated by the payload 
fi-^st opposes the motion, but after the initial overshoot, 
supports the desired motion; while in the second case, the gravity 
first acts in the direction of the desired motion, but after the 
angular displacement exceeds -90 degrees, gravity of the payload 
opposes the motion. Notice, for example, that in the first case 
after .2 seconds, the arm reaches an angle of 4-35 degrees while 
in the second case, the angle is -57 degrees. 

Simultaneous Control of Both Arms: - The extension of the 

control system from a single arm to both arms simultaneously is 
straightforward. The same type of control system is applied to 
both joints, the only difference being the magnitudes of the 
gains. Figure IJ slows the block diagram for the two-link system 
in a form suitable for direct translation into the subroutine 
SfATE as required by SLAM. (Note that this would also be suitable 
for being programmed on an analog computer.) The quantities 
and ♦ represent the commanded angles. Th otli»r quar t it ■ 


» • V 






4 


r 

I 




correspond to the symbols as used in the section "Lagrange's 
Method for Two-Link System." Figures 11 through 14 show time 
history responses of the two controlled angles for various 
combinations of step inputs. 

Note that Figure 13 and Figure 14 illustrate the motion of 
the robot moving the payload from the same initial position 

(x = ; y = 0) 

to the same position 

(x = + ’ y = o) 


in Figure 13 such that at the terminal position = +45**; 

^2 ~ -90**; while in Figure 14, the terminal position is = -45”, 
*2 = +90**. Note the strongly asymmetrical motion, particularly of 
the angle 4*^. 

Figure 14 shows the motion of the payload in the x-y plane as 
function of time for the conditions shown in Figure 13. 


A Control System with Gains Adjusted by 
A Finite-State Machine 

Description of the Gain Adjuster : - As it was stated in the 
section "The Different Control Schemes," we proposed a new 
approach to solve the robot control problem by modifying some of 
the feedback gains based on the state of a finite-state machine 
rather than based on the observed difference btween the robots 
actual response and the response of the refereitce model. 

We will first demonstrate the need for gain adjustment. 

Cnnc,iripr the case where a,,, \ ^ t i « 11 v /..i. 

want tc move the payload from v . . v -i - r. •. .. . • 
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both arms are extended to some oL.^er poinL on the x-axi^, 
either by: 


I c I ' n c ( 


(case 1) 


♦l = -45® = +90® (case 2) 

(Compare with Figure 14, showing the motion for the first case). 

Figure 15 shows the response ( 4 .^ only) for fixed gains. The 
difference in the response between the two cases is due to the 
asymmetry of the moments due to gravity and therefore becomes more 
pronounced the heavier the payload is. 

Only very limited time remained under this contract to 
develop the finite-state machine gain adjuster. It was therefore 
decided to adjust only the gains for the controller of the upper 
arm ( 4 >j). This is the angle which is more difficult to control 
because the moment of inertia about the shoulder joint depends on 
the elbow angle. In the following examples, the gains for the 
controller of (i)^ will remain constant. 

A state-output finite-state machine was devised to set the 
gains. The machine consists of eight states, associated with each 
state are two gain values, one for Kp and one for Kp 
according to the following table: ^ ^ 


State Kp Ko 

Number 1 >'1 

1 600 0.3 

2 600 0.2 

3 800 0.15 

4 800 0.1 

5 300 0.4 

6 300 0.2 
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The input alphabet to the finite-state machine consists of eight 
symbols, the integers 1 through 8. Th input symbol to the FSM is 

if sign positive 

otherwi se 

if \ \ ^ 

d T h 

otherwise ‘ 

If Uj I > 

otherwi se 

Then, the input symbol to the FSM is defined as 

INSYMB = 8 - (4 * IN3 + 2 * IN2 + INI) 

The state transition logic is such that no matter in which 
state the FSM is at the time of receipt of an input symbol, the 
FSM will transit into the state with the same nu'^her as the input 
symbol . 

We realize, of course, that the logic of such a finite-state 
machine gain adjuster is very simple (it amounts to a table look- 
up). Two remarks, however, are appropriate. First, the few 
examples of responses shown in the next section show that the 
system works significantly better than the one previously 
described with fixed gains. 

Second, the intent was to have a more sophisticated FSM to 

adjust the gains. A first improvement would expand the input 

alphabet such that it contains information about the angle (^2 

and maybe the angular rate '^ 2 - Since the moment of inertia 

about the shoulder joint increases with decreasing angle 4 ' 2 » it 

seems desirable to increase K with decreasim) angle 

P <- 

It is clear that the design of the finite-state machine gain 
adjuster, as the input alphabet size increases, becoiicf. ;’.c c ,r ' 


determined as follows: 

Let IN3 = 1 


Let IN2 


1 

0 


Let INI 


1 

0 
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more difficult and can no >nger be achieved by mere intuiLion 
of the designer. When the point of complexity which exceeds the 
intuitive method of designing a FSM gain adjuster is reached, it 
would be beneficial to incorporate evolutionary programming into 
the control system design. The evolutionary program could be 
started with the final "best" machine found by intuition and 
analysis. This machine would serve as the parent machine for 
the evolutionary process. 

The evolution of the FSM gain adjuster could now be 
performed on-line, in real-time. Thus, the FSM gain adjuster 
would not only take care of the time varying dynamic properties 
(such as changing moments of inertia) of the manipulator, but it 
would also compensate for unknown physical parameters of the 
manipulator. With today's available computer resources, such an 
evolutionary control system design appears to be technically 
feasible. It might well provide a solution to the problem of 
designing truly adaptive, multipurpose robots operating in an 
unknown environment. 

Sample Responses: - In Figures 13 and 14 the system 
responses were shown for the two cases: 

1. = +45° = -90° 

2. = -45° = +90° 

Figures 16 and 17 show the system response for the same two cases, 
but having the gains adjusted with the FSM gain adjuster. Not 
only is the response smoother and faster, but much more symme- 
trical when using the FSM gain adjuster. These two examples 
clearly show the superiority of the system with adjusted gains 
over a system with constant gains. 

A last example (Figure 18) shows the system response for a 
ramp input. Even though the system was designed with step 
inputs in mind, the response Lo i amp inputs is quite id L i s f ac to ry . 



or F 






L 


A complete listing of the simulation source program is 
presented in Figure 19. 


CONCLUSIONS 


Robots, whose angles between individual links are controlled 
by servo motors, can be controlled to yield fast response and 
remain stable over the entire operating envelope. This may be 
achieved by control system employing proportional, integral and 
rate feedback, in which some or all of the gains are adjusted 
by finite-state machines. These FSM’s are "driven" by the 
systems past response to commanded inputs. 

Such control systems might offer advantages over adaptive 
model -ref erenc i ng systems because in a digitally controlled robot, 
finite-state machines are easily integrated with the remainder of 
the control system. 

It would seem worthwhile to implement the control system, as 
described in this report, in actual hardware and to compare its 
performance with the one of the simulation. 
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TABLE I J 

1 

SELECTED PUMA SPECIFICATIONS 


Data Provided by Unimation, Inc. 

Degrees of Freedom: 5 rotational 

no translational 

Rotatonal Limits 


(1) 

Waist: 

+ 160*' 

(2) 

Shouder : 

+ 165° 

(3) 

E 1 bow : 

+ 135° 

(4) 

Wrist: 

+ 105° 

(5) 

Joint 5 : 

+ 180° 


Maximum Static Force 
at the "hand": 


Maximum Payload: 

Maximum Hand Acceleration: 
Maximum Hand Velocity: 
Control : 


58 N (13 lb.) 

223 N (5 lb.) 

1 g 

1.0 m/s (3.3 feet/s) 
Electric DC Servomotors 


Data Obtained from Measurements at LRC 


Maximum torque in elbow joint 
(averaged between up and down motion) 

‘^m IX 


Maximum torque in 
(averaqed between 


shoul der joint 
up and down motion' 


M 


ir ) X 


163 Nm 


244 Nm 
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TABLE 2 

Comparison between Newton/Euler and Lagrange Method. 





NEWTON/EULER 

1 

LAGRANGE 

1 

1 


■ 



• More "visible" correlation 

1 

• SEUiNG UP EQUATIONS | 


BETWEEN EQUATIONS OF MOTION 

RELATIVELY STRAIGHT FORWARD i 

ADVANTAGES 

AND PHYSICAL SYSTEM 

i 


• Internal (reaction) Forces 

• Only minimum amount of i 


AND Movements 

1 

EQUATIONS REQUIRED i 

i 

^ _ _ J 


• Formulation and solution 

• Solve for highest derivatives j 

iDISADVANTAGES 

OF constraint equations is 

OF EACH state variable j 

1 

I 

1 

1 ; 

TRICKY 

t 

1 

I 

t 

) 

• Likelihood of sign errors 

• Does not provide reaction 1 

L 

HIGH 

FORCES AND MOMENTS ‘ 

_ _ J 
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Figure 9: Responses of Lower Arm Lo f.tep 

Inputs of Varying Vnnnitudes and Sic^'- 
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Figure 17: System Response inth Finito-StU 

Machine Gain Adjuster for 


' 1 . 


-45" 


?C = -90 
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(Compare with Figure 14) 











GEN, BURGIN. ROBOT, 2/ 12/82; 
INIT1ALIZE,0,25; 

CONTINUOUS, 6. 0,0. 0005.0. 1,0. 1. W 
RECORD. TNOW, TIME, ,T, oil; 
V0R,XX(6) ,PHI1 ; 

•VAR,XX(3) ,PHI2; 

VAR.XX(2l),M0Mt; 

VAR,XX(20) ,M0M2; 

*/AR,XX(31 ), STATE; 

SIMULATE; 

I IN; 

END OF FILE 


./ 
! i 


/get, roDc»tS 


C; 

C 


c 

c 

c 

c 


c 


PROGRAM MAIN( T APES , TAPE6=80 . INPUT . OUTPUT , TAPE 1 /, TAPE 1 8 ) 

DIMENSION NSET(2000) 

COMMON OSET(ISOO) 

E0U1VAEENCE(0SET( 1 ) ,NSET( 1 ) ) 

COMMON/5COM1/ ATRIB( 100) ,DD? 100) .DDL( 100) . DTNOW. II .MFA.MSTOP.NCLNR 
1 .NCRDR.NPRNT.NNRUN.NNSET.NTAPE.SSl 1 00 ) . SSL (1 00 ) , TNEXT . TNOW . XX ( 1 00 ) 

COMMON/BURG IN 1 / AA . BB . CC , DDD . G , P I 

CnMMUN/BURGIN2/L1 .L2.M1 .M2.MP.il .I2,R1 .R2.M0M1 .M0M2.CF 1 .CF2 
REAL LI ,L2.M1 .M2.MP,I1 . 12. MOM 1 ,M0M2 

COMMON/CON TROL /RS , KF . KP . KR , K I 
COMMON/CNTRLl /RSI ,KF1 ,KP1 .KR1 .KI 1 
REAL KF.KP.KR.KI 
REAL KFl ,KP1 .KRl ,KI1 


NNSFT=1500 

NCRDR=5 

NPRNT-F. 

NIAPE=18 

NNSET=1500 


REWIND 5 
REWIND 6 
RFWIND 17 
REWIND 18 
PI-^.-^ATANI 1 




Figure 19: 
Computer 
Source 


Listing of 
Simulation 
Program 


C 
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noo 


1 continue: 

REWIND S 
cm I SLAM 
STOf’ 

END 

SUBROUTINE STATE 


.i ^ 
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C 

C 

C 

C 


C 

C 

c 


c 

c 


c 

c 


DIMENSION NSET(ISOO) 

COMMON QSETU500) 
E0UIV/ALENCE(0SET(1) ,NSlT(1) ) 


COMMON /SC0M1 / ATRIB ( 100 > , DD ( 1 00 ) . DDL (100) , DTNOW, 1 1 ,MFA .MSTOP.NCLNR 
1 , NCRDR , NPRNT , NNRUN . NNSE T , N TAPE , SS (1 00 ) , SSL (1 00 ) i TNEXT . TNOW, XX ( 1 00 ) 

C0MMCIN/BURG1N1/AA.BB,CC,DDD,G.PI 


C0MM0N/DURGIN2/L1 .L2.M1 ,M2,MP. II , 1 2 . R 1 , R2 , MOM 1 , M0M2 . CF 1 , CF2 
REAL LM 2,M1 ,M2,MP,I1 ,I2,MQMI ,MOM2 

COMMON/CONTROL /RS . KF , KP . KR , K I 
COMMON/CNFRL1/RS1 .KF 1 ,KP1 .KR1 ,KI 1 
REAL KF.KP.KR.KI 
REAL KF1 ,KP1 ,KR1 ,KI1 

EOUIVm f NCE(SS( 1 ) ,PHI1 ) . ( SS ( 2 ) , PH1 1 DOT ) , ( SS ( 3 ) , ?HI2 ) , 

+ (SS(^) .PH12D0T) 

DA I A Slope 1/0. 1S48S7/ 


If I INOW.L T.S. )G0 TO 710 
IF ( TNOW.LT . IS. )GO TO 720 
If ( INOW.L T.20. )G0 TO 730 
GO TO 740 

710 RSP INOW^SlOPEI 
GO 10 7S0 

720 RSI =0. 7/429- (TNOW-S. )»SL0PE1 
GO 10 /SO 

730 RSI ^ 0. 7/420+( INOW- IS. )^Sl OPE 1 
GO 10 7S0 

740 RS1=0. 

7S0 CON! I NUT 


SINPflI2 = SIN(PHI2) 

G1)SPIII2 = CUS(PHI2) 

CDf.Pfin =COS(FH1 1 ) 

SINPHII =SIN(PHI1 ) 

COS !P2^ COS (PH II »PHI2) 

SIN1P2- SINCPHIl ♦f'MI'M 

t i y u I'e 1 v'Cun . u , 


m r A nn^cc+r. *(;b*(:osp»u 2 

DL lA =CC*BB»C0SPHI2 
Of 1 TiVCC 

(AMMO PHI2D0T M2. «PfU IDOi »PM L DO I ) ^S’fMS INI ML’ 1MM)»(. •• USf'ili 
♦ -BL>G»C0S1P2/L1 

FPSIL0N--BB«G»C0SIP2/L1 - PHI 1DOT ’»»2«DD’»SINI HI2 
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c 


CALL GAINS 


cn.^- ... ?s 

OF POC;^ QUALITY 


ERR0R=RS-K?»PHI?-KK^»PHI2D01 
ERROR I =RS1 -KPHPH1 1 -KR 1 «PH1 1 DOT 
M0M1=KF1»ERR0R1+K11*SS(C.) 

IF(M0m .GT.244. )MC111-2AA. 

IF(H0M1 .LT.-24A. )M0M1=-2AA. 

MOM2=KF^»ERROR+KI«SS(b) 

IF(M0M2.GT. 163. )M0rt2=163. 

IF (M0M2.lt. -163. )M0M2--163. 

DD(1 )-SS(2) 

DEN0M = [)ELTA*ALFA-BeTM^^2 

DD( 2 ) = ( DELTA* ( GAMMA + M0M1 ) -BE TA* (EPSIL0N + M0M2 ) ) /DENUM 
DD(3)=SS(A) 

DD( 4 ) = ( -BETA* ( GAMMA ^MOMl ) +ALFA* (EPSIL0N+M0M2 ) ) /DENOM 

DD(S)=ERR0R 

DD(6)=ERR0R1 

XX(3)=S5(3)*XX(S7) 

XX(4)=SS(A)*XX(57) 

XX(5)=DD(A)*XX(B7) 

XX(6)=SS( 1 )*XX(B7) 

XX(7)=L1*COSPHI1*100. 

XX(8)=L1*SINPHI1*100. 

XX(9)=XX( 7)+l2*C0S1P2*100. 

XX( 10)=XX(8)+L?*SIN1P2*IOO. 

XX(20)=M0M2 
XX(21 )=M0M1 
RETURN 

end Figure 19 (Cont'd) 

SUBROUTINE INTLC 


DIMENSION NSET(ISOO) 

COMMON OSET(IBOO) 

E0UIVALENCE(0SET(1 ) .NSET( 1 ) ) 

C0MM0N/SC0M1/ ATR IB( 1 00 ) , DIM 1 OU ) , DDL ( 1 00 ) .DTNOW, II,MFA.MSTOP.NCLNR 
1 ,NCRDR,NPRNT.NNRUN,NNSET,NTAPE ,SS( 100) .SSL(IOO) ,TNEXT.TNOW,XX(100) 

COMMON /BURG IN 1 /AA , BB , CC . DDD , G , P I 

C:0MM0N/BURGIN2/L1 .L2.M1 , M2 . MP , 1 1 . 1 2 , R1 , R2 . MOM 1 , M0M2 ,CF 1 .CF2 
REAL LI ,L2.M1 ,M2,MP, II , I2,M0M1 .MOM2 

COMMON/CON TROL /RS , KF , KP . KR . K I 
COMMOM/CNTRLI/RSI , KF 1 . KP 1 , KR 1 . M 1 
RFAL KF1 ,KR1 .KI1 .KPl 
REAL KF,KP,KR,KI 


EOUIOAl F N(.E(SSv 1 ) .PHI 1 ) . » SS( . ■ , tU i I'OI ) . ( .-.S (3 ) .r’hL’ . . 
+ (SS(^n .PHl2t)0n 

PI-4. '»ATAN( 1 . ) 

XX(B7)- 180. /PI 

«♦* GLOME 1RICAL AND PHYSICAL DAI A 


L 

C 


L1=0.42 

L2=0.42 

Ml=16.95 

M2=16.95 

MP=2.9 

11=0.9 

12=0.5 

R1=0.17 

R2=0.17 

M0t11=0. 

liOM2 = 0. 

G=9.81 

CF1=0. 

KF=50. 

KR=0.4 
KP=1 . 
kI=2U0. 

RS - -no. /XX(S7 ) 

KF 1-200. 

KRl-0.2 

i<P 1 = 1 . 

K1 1=800- 
C 

RSI = /XX(5/) 

CF 2=0. 

C 

C INITIAL CONDITIONS 


OF pc : 


r* r 




--7 

; L.TY 


c 

(' 

V.. 


c 




c 

c 


ss n ) = 0 . 

SS(2) 0. 
SS(3)-0. 

ss ( 4 ) = 0 . 

SS(S)=0. 

SS(o)=0. 


Figure 19 (Ccnt'd) 


00 = 1 1 + M1 *R1 >^?+l'12»L1 *^2+MP»L 1 »-*2 
8L!= (M2’*R2 + MP»L2)»L1 
CC=I2 + M2»R2«-*2 + rtP»L2*»2 
DDD=m »R1 + (M2 + fiP)*Ll 

PR1N1 91 ,riP.KF.KI,KP,KR 

91 FORMOK///" MP KF KI KP KR = " . 9F 1 0 . 2 , F 1 0 . ^ . /// ) 
PRINT 92,KF1 ,KI1 .KP1 ,KR1 

92 FORMOK/," KF 1 KIl KP1 KR1 = " , 3F 1 0 . 2 , F 1 0 . 0 , // ) 

RF.TURN 

LMD 


SUDROUTINE COINS 


C 

DIMlNSIGN NSEF(5000) 

( OMMON OSEKSOOO) 

F UOlOOl ! NCECFjSEK 1 ) ,NSF 1(1)) 

c 

CGMMON/SCOMI/ OTRIB( 1 00) ,DD( 100) .DDL ( I 00) DINOW, 1 I ,MF NR 


C-D2 


i 


c 

c 

c 


c 

c 


c 


1 ,NCRDR.NPRNT,NNRUN,NNStT.Nlrt.T,SS( 1 00 ) . S-.L (1 00 ) , TNtX i , VNO^i . XX( 1 00 ) 
COMMON/BURG IN I /AA BB , CC , ODD , G , P I 

COMMON/BURG IN2/L 1 . L2 , M 1 , M? . MP , 1 1 . 12 , R t , R2 , MOfI 1 , MOM2 , CF 1 , CF2 
real Li ,L2,I11 .M2,I1P,I1 .I2,M0M1 ,(10M2 


C0f1l10N/C0N TROL /RS . KF , KP , KR , K I 
C0MM0N/CNTRL1 /RSI ,KF1 .KPl .KRt ,KI t 
REAL KF1 ,KR1 ,KI1 
REAL KF,KP,KR,KI 


F.r:.:" i3 
OF POOR QUALITY 


E0UI0ALENCE(SS( t ) ,PHI1 ) , (SS(2) .PHI 
+ (SS(4) .PHI2D0T) 


IDOT) , (SS(3) ,PHI2) 


♦ 


DATA PHI1DMX/0.78S3/,ILAST/0/ 

ERR1=RS1-PHI1 

IN3=1 

IF(ERR1 .LE.O. )IN3=0 
IN2=1 

IF(ABS(ERR1 ) .LT.PI/4. )IN2=0 

INI = 1 

IF(ABS(PHI1DOT) .LT.O. 7853)IN1=0 


INSY:1B = 8- (^»IN3 + 2*IN2 + IN1 ) 

^ GO IO( 1 1 00 . 1 200 . 1 300, MOO, 1 SOO , 1600 , 1 700 , 1 800) INSYMB 
1100 INEW=1 

IFdNEN.NE.ILAST )CALL PRTl ( TNOW, INEW, ILAST ) 

Kr I ~ oO 0 • 

KR1=0.3 
GO TO 999 
1200 IN£W=2 

It- ( INEW.NE. ILASDCALL PRT I ( TNOW . IMEN . IL AST ) 

KF 1 -600 . 

KR1=0. 1 
GO TO 999 
1300 INEW-3 

IFJINEW.NE. ILASDCALL PRTK TNOW. INEW, ILAST) 

Kf 1=800 . 

KR1=0. IS 
GO TO 999 
MOO IN£W = ^ 

IF (INEW.NE. ILASDCALL PR TI ( TNOW. INEW. ILAST) 

KF 1=800, 

KR1=0. 1 
GO TO 999 
1S00 INEW=S 

IF ( INEW.NE . ILASDCALL PRT I ( TNOW. INEW. ILAST) 

KF 1=300. 

KRD0.4 
GO TO 999 
1600 INEW=6 

IF (INEW.NE. ILAST )CALL PRTK TNOW. INEW, ILAST) 

KF 1 = 300. 

KRM0.2 
GO TO 999 

1 700 INEW=7 Figure 

IF ( TNEW.NE . ILASDCALL PRTK TNOW, INEW, ILAST) 

KF 1 =S00. 


19 (Cent ' d ) 
C-53 


1 


ncj 


Cr ?rC7> C JAJJTY 

KRl-0.3 
GO TO 999 
1800 INEW=8 

ir (INEW.NE. ILAST)CftLL PRTKTNOW. INEW, JLAST) 
KF 1=500. 

KR1=0.2 
999 CONTINUE 
XX (31 )=TNEW 
END 


SUBROUTINE PRTI ( TNOW, INEW. ILAST ) 



PRINT 91 , TNOW, ILAST, INEW 

91 FORMATC STATE CHANGED AT TIME "F12.3" FROM"!^" T0"I4) 
ILAST=INEW 
RETURN 
END 


END OF FILE 


Figure 19 (Cont'd) 


C-54 


